English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

LiDAR and camera are two important sensors for 3D object detection inautonomous driving. Despite the increasing popularity of sensor fusion in thisfield, the robustness against inferior image conditions, e.g., bad illuminationand sensor misalignment, is under-explored. Existing fusion methods are easilyaffected by such conditions, mainly due to a hard association of LiDAR pointsand image pixels, established by calibration matrices. We propose TransFusion,a robust solution to LiDAR-camera fusion with a soft-association mechanism tohandle inferior image conditions. Specifically, our TransFusion consists ofconvolutional backbones and a detection head based on a transformer decoder.The first layer of the decoder predicts initial bounding boxes from a LiDARpoint cloud using a sparse set of object queries, and its second decoder layeradaptively fuses the object queries with useful image features, leveraging bothspatial and contextual relationships. The attention mechanism of thetransformer enables our model to adaptively determine where and whatinformation should be taken from the image, leading to a robust and effectivefusion strategy. We additionally design an image-guided query initializationstrategy to deal with objects that are difficult to detect in point clouds.TransFusion achieves state-of-the-art performance on large-scale datasets. Weprovide extensive experiments to demonstrate its robustness against degeneratedimage quality and calibration errors. We also extend the proposed method to the3D tracking task and achieve the 1st place in the leaderboard of nuScenestracking, showing its effectiveness and generalization capability.

TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers