English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advancements in multimodal pre-training methods have shown promisingefficacy in 3D representation learning by aligning multimodal features across3D shapes, their 2D counterparts, and language descriptions. However, themethods used by existing multimodal pre-training frameworks to gathermultimodal data for 3D applications lack scalability and comprehensiveness,potentially constraining the full potential of multimodal learning. The mainbottleneck lies in the language modality's scalability and comprehensiveness.To address this, we introduce ULIP-2, a tri-modal pre-training framework thatleverages state-of-the-art large multimodal models to automatically generateholistic language counterparts for 3D objects. It does not require any 3Dannotations, and is therefore scalable to large datasets. We conductexperiments on two large-scale 3D datasets, Objaverse and ShapeNet, and augmentthem with tri-modal datasets of 3D point clouds, images, and language fortraining ULIP-2. ULIP-2 achieves significant improvements on downstreamzero-shot classification on ModelNet40 (74.0% in top-1 accuracy); on thereal-world ScanObjectNN benchmark, it obtains 91.5% in overall accuracy withonly 1.4 million parameters, signifying a breakthrough in scalable multimodal3D representation learning without human 3D annotations. The code, along withthe generated tri-modal datasets, can be found athttps://github.com/salesforce/ULIP.

ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding