English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision-and-language (VL) pre-training, which aims to learn a generalrepresentation of image-text pairs that can be transferred to variousvision-and-language tasks. Compared with modeling uni-modal data, the mainchallenge of the VL model is: how to learn the cross-modal interaction frommultimodal data, especially the fine-grained interaction. Existing works haveshown that fully transformer-based models that adopt attention mechanisms tolearn in-layer cross-model interaction can demonstrate impressive performanceon various cross-modal downstream tasks. However, they ignored that thesemantic information of the different modals at the same layer was not uniform,which leads to the cross-modal interaction collapsing into a limitedmulti-modal semantic information interaction. In this work, we propose theUNIMO-3 model, which has the capacity to simultaneously learn the multimodalin-layer interaction and cross-layer interaction. UNIMO-3 model can establisheffective connections between different layers in a cross-modal encoder, andadaptively capture the interaction between two modalities at different levels.The experimental results show that our model achieves state-of-the-artperformance in various downstream tasks, and through ablation study can provethat effective cross-layer learning improves the model's ability of multimodalrepresentation.

UNIMO-3: Multi-granularity Interaction for Vision-Language Representation Learning