English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Masked Autoencoders (MAE) play a pivotal role in learning potentrepresentations, delivering outstanding results across various 3D perceptiontasks essential for autonomous driving. In real-world driving scenarios, it'scommonplace to deploy multiple sensors for comprehensive environmentperception. While integrating multi-modal features from these sensors canproduce rich and powerful features, there is a noticeable gap in MAE methodsaddressing this integration. This research delves into multi-modal MaskedAutoencoders tailored for a unified representation space in autonomous driving,aiming to pioneer a more efficient fusion of two distinct modalities. Tointricately marry the semantics inherent in images with the geometricintricacies of LiDAR point clouds, the UniM$^2$AE is proposed. This modelstands as a potent yet straightforward, multi-modal self-supervisedpre-training framework, mainly consisting of two designs. First, it projectsthe features from both modalities into a cohesive 3D volume space, ingeniouslyexpanded from the bird's eye view (BEV) to include the height dimension. Theextension makes it possible to back-project the informative features, obtainedby fusing features from both modalities, into their native modalities toreconstruct the multiple masked inputs. Second, the Multi-modal 3D InteractiveModule (MMIM) is invoked to facilitate the efficient inter-modal interactionduring the interaction process. Extensive experiments conducted on the nuScenesDataset attest to the efficacy of UniM$^2$AE, indicating enhancements in 3Dobject detection and BEV map segmentation by 1.2\%(NDS) and 6.5\% (mIoU),respectively. Code is available at https://github.com/hollow-503/UniM2AE.

UniM$^2$AE: Multi-modal Masked Autoencoders with Unified 3D Representation for 3D Perception in Autonomous Driving