English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Traditional LiDAR-based object detection research primarily focuses onclosed-set scenarios, which falls short in complex real-world applications.Directly transferring existing 2D open-vocabulary models with some known LiDARclasses for open-vocabulary ability, however, tends to suffer from over-fittingproblems: The obtained model will detect the known objects, even presented witha novel category. In this paper, we propose OpenSight, a more advanced 2D-3Dmodeling framework for LiDAR-based open-vocabulary detection. OpenSightutilizes 2D-3D geometric priors for the initial discernment and localization ofgeneric objects, followed by a more specific semantic interpretation of thedetected objects. The process begins by generating 2D boxes for generic objectsfrom the accompanying camera images of LiDAR. These 2D boxes, together withLiDAR points, are then lifted back into the LiDAR space to estimatecorresponding 3D boxes. For better generic object perception, our frameworkintegrates both temporal and spatial-aware constraints. Temporal awarenesscorrelates the predicted 3D boxes across consecutive timestamps, recalibratingthe missed or inaccurate boxes. The spatial awareness randomly places some``precisely'' estimated 3D boxes at varying distances, increasing thevisibility of generic objects. To interpret the specific semantics of detectedobjects, we develop a cross-modal alignment and fusion module to first align 3Dfeatures with 2D image embeddings and then fuse the aligned 3D-2D features forsemantic decoding. Our experiments indicate that our method establishesstate-of-the-art open-vocabulary performance on widely used 3D detectionbenchmarks and effectively identifies objects for new categories of interest.

OpenSight: A Simple Open-Vocabulary Framework for LiDAR-Based Object Detection