English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper introduces a novel self-supervised learning framework forenhancing 3D perception in autonomous driving scenes. Specifically, ourapproach, named NCLR, focuses on 2D-3D neural calibration, a novel pretext taskthat estimates the rigid transformation aligning camera and LiDAR coordinatesystems. First, we propose the learnable transformation alignment to bridge thedomain gap between image and point cloud data, converting features into aunified representation space for effective comparison and matching. Second, weidentify the overlapping area between the image and point cloud with the fusedfeatures. Third, we establish dense 2D-3D correspondences to estimate the rigidtransformation. The framework not only learns fine-grained matching from pointsto pixels but also achieves alignment of the image and point cloud at aholistic level, understanding their relative pose. We demonstrate NCLR'sefficacy by applying the pre-trained backbone to downstream tasks, such asLiDAR-based 3D semantic segmentation, object detection, and panopticsegmentation. Comprehensive experiments on various datasets illustrate thesuperiority of NCLR over existing self-supervised methods. The results confirmthat joint learning from different modalities significantly enhances thenetwork's understanding abilities and effectiveness of learned representation.Code will be available at \url{https://github.com/Eaphan/NCLR}.

Self-supervised Learning of LiDAR 3D Point Clouds via 2D-3D Neural Calibration