English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The video-language (VL) pretraining has achieved remarkable improvement inmultiple downstream tasks. However, the current VL pretraining framework ishard to extend to multiple modalities (N modalities, N>=3) beyond vision andlanguage. We thus propose LanguageBind, taking the language as the bind acrossdifferent modalities because the language modality is well-explored andcontains rich semantics. Specifically, we freeze the language encoder acquiredby VL pretraining, then train encoders for other modalities with contrastivelearning. As a result, all modalities are mapped to a shared feature space,implementing multi-modal semantic alignment. While LanguageBind ensures that wecan extend VL modalities to N modalities, we also need a high-quality datasetwith alignment data pairs centered on language. We thus propose VIDAL-10M withVideo, Infrared, Depth, Audio and their corresponding Language, naming asVIDAL-10M. In our VIDAL-10M, all videos are from short video platforms withcomplete semantics rather than truncated segments from long videos, and all thevideo, depth, infrared, and audio modalities are aligned to their textualdescriptions. LanguageBind has achieved superior performance on a wide range of15 benchmarks covering video, audio, depth, and infrared. Moreover, multipleexperiments have provided evidence for the effectiveness of LanguageBind inachieving indirect alignment and complementarity among diverse modalities. Codeaddress: https://github.com/PKU-YuanGroup/LanguageBind

LanguageBind: Extending Video-Language Pretraining to N-modality by Language-based Semantic Alignment