English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The rising importance of 3D understanding, pivotal in computer vision,autonomous driving, and robotics, is evident. However, a prevailing trend,which straightforwardly resorted to transferring 2D alignment strategies to the3D domain, encounters three distinct challenges: (1) Information Degradation:This arises from the alignment of 3D data with mere single-view 2D images andgeneric texts, neglecting the need for multi-view images and detailedsubcategory texts. (2) Insufficient Synergy: These strategies align 3Drepresentations to image and text features individually, hampering the overalloptimization for 3D models. (3) Underutilization: The fine-grained informationinherent in the learned representations is often not fully exploited,indicating a potential loss in detail. To address these issues, we introduceJM3D, a comprehensive approach integrating point cloud, text, and image. Keycontributions include the Structured Multimodal Organizer (SMO), enrichingvision-language representation with multiple views and hierarchical text, andthe Joint Multi-modal Alignment (JMA), combining language understanding withvisual representation. Our advanced model, JM3D-LLM, marries 3D representationwith large language models via efficient fine-tuning. Evaluations on ModelNet40and ScanObjectNN establish JM3D's superiority. The superior performance ofJM3D-LLM further underscores the effectiveness of our representation transferapproach. Our code and models are available at https://github.com/Mr-Neko/JM3D.

JM3D & JM3D-LLM: Elevating 3D Understanding with Joint Multi-modal Cues