TransCalib: Automated Extrinsic Calibration of LiDAR–Camera Fusion using Convolutional Transformer for Targetless Self-Alignment

In autonomous systems and robotic applications, accurate extrinsic calibration between light detection and ranging (LiDAR) sensors and cameras is crucial for reliable sensor fusion. Several techniques have been developed, including target-based and targetless calibration, but they are either impractical for real-world applications or limited in extracting complex and diverse features. This study presents TransCalib, an innovative deep-learning method for targetless and automatic extrinsic calibration. TransCalib predicts the misalignment between the camera and LiDAR by leveraging EfficientNetV2 to obtain features from the RGB camera image and LiDAR point cloud projection image (depth image), owing to its performance and parameter efficiency.We also developed an innovative feature-matching module that comprises a calibration convolutional feature aggregation block (Calib-CFAB) and a convolutional self-attention (CSA) transformer. Calib-CFAB enriches the combined feature map of the RGB and depth images, while the CSA transformer obtains the correlation in the feature maps. Trained and tested on the KITTI odometry dataset, TransCalib achieved a mean absolute rotation error of 0.14° and a mean translation error of 1.8 cm, outperforming existing methods. The proposed method allows for a robust fusion of LiDAR and camera data, improving the perception abilities of autonomous systems.