English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Open-vocabulary 3D instance segmentation has emerged as a frontier topic dueto its capability to segment 3D instances beyond a predefined set ofcategories. However, compared to significant progress in the 2D domain, methodsfor 3D open-vocabulary instance segmentation are hindered by the limited scaleof high-quality annotated 3D data. To harness the capabilities of 2D models,recent efforts have focused on merging 2D masks based on metrics such asgeometric and semantic similarity to form 3D instances. In contrast to theselocal metrics, we propose a novel metric called view consensus to betterexploit multi-view observation. The key insight is that two 2D masks should beconsidered as belonging to the same instance if a considerable number of other2D masks from other views contain both these two masks. Based on this metric,we build a global mask graph and iteratively cluster masks, prioritizing maskpairs with solid view consensus. The corresponding 3D points cluster of these2D mask clusters can be regarded as 3D instances, along with the fusedopen-vocabulary features from clustered 2D masks. Through this multi-viewverification and fusion mechanism, our method effectively leverages the priorinstance knowledge from massive 2D masks predicted by visual foundation models,eliminating the need for training on 3D data. Experiments on publicly availabledatasets, including ScanNet200 and MatterPort3D, demonstrate that our methodachieves state-of-the-art performance in both open-vocabulary instancesegmentation and class-agnostic mask generation. Our project page is athttps://pku-epic.github.io/MaskClustering.

MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation