English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Vision-Language (VL) models with the Two-Tower architecture have dominatedvisual-language representation learning in recent years. Current VL modelseither use lightweight uni-modal encoders and learn to extract, align and fuseboth modalities simultaneously in a deep cross-modal encoder, or feed thelast-layer uni-modal representations from the deep pre-trained uni-modalencoders into the top cross-modal encoder. Both approaches potentially restrictvision-language representation learning and limit model performance. In thispaper, we propose BridgeTower, which introduces multiple bridge layers thatbuild a connection between the top layers of uni-modal encoders and each layerof the cross-modal encoder. This enables effective bottom-up cross-modalalignment and fusion between visual and textual representations of differentsemantic levels of pre-trained uni-modal encoders in the cross-modal encoder.Pre-trained with only 4M images, BridgeTower achieves state-of-the-artperformance on various downstream vision-language tasks. In particular, on theVQAv2 test-std set, BridgeTower achieves an accuracy of 78.73%, outperformingthe previous state-of-the-art model METER by 1.09% with the same pre-trainingdata and almost negligible additional parameters and computational costs.Notably, when further scaling the model, BridgeTower achieves an accuracy of81.15%, surpassing models that are pre-trained on orders-of-magnitude largerdatasets. Code and checkpoints are available athttps://github.com/microsoft/BridgeTower.

BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning