English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we focus on developing knowledge distillation (KD) for compact3D detectors. We observe that off-the-shelf KD methods manifest their efficacyonly when the teacher model and student counterpart share similar intermediatefeature representations. This might explain why they are less effective inbuilding extreme-compact 3D detectors where significant representationdisparity arises due primarily to the intrinsic sparsity and irregularity in 3Dpoint clouds. This paper presents a novel representation disparity-awaredistillation (RDD) method to address the representation disparity issue andreduce performance gap between compact students and over-parameterizedteachers. This is accomplished by building our RDD from an innovativeperspective of information bottleneck (IB), which can effectively minimize thedisparity of proposal region pairs from student and teacher in features andlogits. Extensive experiments are performed to demonstrate the superiority ofour RDD over existing KD methods. For example, our RDD increases mAP ofCP-Voxel-S to 57.1% on nuScenes dataset, which even surpasses teacherperformance while taking up only 42% FLOPs.

Representation Disparity-aware Distillation for 3D Object Detection