English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Existing vision-language pre-training (VLP) methods primarily rely on pairedimage-text datasets, which are either annotated by enormous human labors, orcrawled from the internet followed by elaborate data cleaning techniques. Toreduce the dependency on well-aligned image-text pairs, it is promising todirectly leverage the large-scale text-only and image-only corpora. This paperproposes a data augmentation method, namely cross-modal CutMix (CMC), forimplicit cross-modal alignment learning in unpaired VLP. Specifically, CMCtransforms natural sentences from the textual view into a multi-modal view,where visually-grounded words in a sentence are randomly replaced by diverseimage patches with similar semantics. There are several appealing proprietiesof the proposed CMC. First, it enhances the data diversity while keeping thesemantic meaning intact for tackling problems where the aligned data arescarce; Second, by attaching cross-modal noise on uni-modal data, it guidesmodels to learn token-level interactions across modalities for betterdenoising. Furthermore, we present a new unpaired VLP method, dubbed asVLMixer, that integrates CMC with contrastive learning to pull together theuni-modal and multi-modal views for better instance-level alignments amongdifferent modalities. Extensive experiments on five downstream tasks show thatVLMixer could surpass previous state-of-the-art unpaired VLP methods.

VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix