English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

3D panoptic segmentation is a challenging perception task, which aims topredict both semantic and instance annotations for 3D points in a scene.Although prior 3D panoptic segmentation approaches have achieved greatperformance on closed-set benchmarks, generalizing to novel categories remainsan open problem. For unseen object categories, 2D open-vocabulary segmentationhas achieved promising results that solely rely on frozen CLIP backbones andensembling multiple classification outputs. However, we find that simplyextending these 2D models to 3D does not achieve good performance due to poorper-mask classification quality on novel categories. In this paper, we proposethe first method to tackle 3D open-vocabulary panoptic segmentation. Our modeltakes advantage of the fusion between learnable LiDAR features and dense frozenvision CLIP features, using a single classification head to make predictionsfor both base and novel classes. To further improve the classificationperformance on novel classes and leverage the CLIP model, we propose two novelloss functions: object-level distillation loss and voxel-level distillationloss. Our experiments on the nuScenes and SemanticKITTI datasets show that ourmethod outperforms strong baselines by a large margin.

3D Open-Vocabulary Panoptic Segmentation with 2D-3D Vision-Language Distillation