English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Current 3D open-vocabulary scene understanding methods mostly utilizewell-aligned 2D images as the bridge to learn 3D features with language.However, applying these approaches becomes challenging in scenarios where 2Dimages are absent. In this work, we introduce a new pipeline, namely,OpenIns3D, which requires no 2D image inputs, for 3D open-vocabulary sceneunderstanding at the instance level. The OpenIns3D framework employs a"Mask-Snap-Lookup" scheme. The "Mask" module learns class-agnostic maskproposals in 3D point clouds. The "Snap" module generates synthetic scene-levelimages at multiple scales and leverages 2D vision language models to extractinteresting objects. The "Lookup" module searches through the outcomes of"Snap" with the help of Mask2Pixel maps, which contain the precisecorrespondence between 3D masks and synthetic images, to assign category namesto the proposed masks. This 2D input-free and flexible approach achievesstate-of-the-art results on a wide range of indoor and outdoor datasets by alarge margin. Moreover, OpenIns3D allows for effortless switching of 2Ddetectors without re-training. When integrated with powerful 2D open-worldmodels such as ODISE and GroundingDINO, excellent results were observed onopen-vocabulary instance segmentation. When integrated with LLM-powered 2Dmodels like LISA, it demonstrates a remarkable capacity to process highlycomplex text queries which require intricate reasoning and world knowledge.Project page: https://zheninghuang.github.io/OpenIns3D/

OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation