English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Open-vocabulary 3D Object Detection (OV-3DDet) aims to detect objects from anarbitrary list of categories within a 3D scene, which remains seldom exploredin the literature. There are primarily two fundamental problems in OV-3DDet,i.e., localizing and classifying novel objects. This paper aims at addressingthe two problems simultaneously via a unified framework, under the condition oflimited base categories. To localize novel 3D objects, we propose an effective3D Novel Object Discovery strategy, which utilizes both the 3D box geometrypriors and 2D semantic open-vocabulary priors to generate pseudo box labels ofthe novel objects. To classify novel object boxes, we further develop across-modal alignment module based on discovered novel boxes, to align featurespaces between 3D point cloud and image/text modalities. Specifically, thealignment process contains a class-agnostic and a class-discriminativealignment, incorporating not only the base objects with annotations but alsothe increasingly discovered novel objects, resulting in an iteratively enhancedalignment. The novel box discovery and crossmodal alignment are jointly learnedto collaboratively benefit each other. The novel object discovery can directlyimpact the cross-modal alignment, while a better feature alignment can, inturn, boost the localization capability, leading to a unified OV-3DDetframework, named CoDA, for simultaneous novel object localization andclassification. Extensive experiments on two challenging datasets (i.e.,SUN-RGBD and ScanNet) demonstrate the effectiveness of our method and also showa significant mAP improvement upon the best-performing alternative method by80%. Codes and pre-trained models are released on the project page.

CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detection