English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

LiDAR and camera are two modalities available for 3D semantic segmentation inautonomous driving. The popular LiDAR-only methods severely suffer frominferior segmentation on small and distant objects due to insufficient laserpoints, while the robust multi-modal solution is under-explored, where weinvestigate three crucial inherent difficulties: modality heterogeneity,limited sensor field of view intersection, and multi-modal data augmentation.We propose a multi-modal 3D semantic segmentation model (MSeg3D) with jointintra-modal feature extraction and inter-modal feature fusion to mitigate themodality heterogeneity. The multi-modal fusion in MSeg3D consists ofgeometry-based feature fusion GF-Phase, cross-modal feature completion, andsemantic-based feature fusion SF-Phase on all visible points. The multi-modaldata augmentation is reinvigorated by applying asymmetric transformations onLiDAR point cloud and multi-camera images individually, which benefits themodel training with diversified augmentation transformations. MSeg3D achievesstate-of-the-art results on nuScenes, Waymo, and SemanticKITTI datasets. Underthe malfunctioning multi-camera input and the multi-frame point clouds input,MSeg3D still shows robustness and improves the LiDAR-only baseline. Our code ispublicly available at \url{https://github.com/jialeli1/lidarseg3d}.

MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving