English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Vision and Language (VL) models offer an effective method for aligningrepresentation spaces of images and text, leading to numerous applications suchas cross-modal retrieval, visual question answering, captioning, and more.However, the aligned image-text spaces learned by all the popular VL models arestill suffering from the so-called `object bias' - their representations behaveas `bags of nouns', mostly ignoring or downsizing the attributes, relations,and states of objects described/appearing in texts/images. Although some greatattempts at fixing these `compositional reasoning' issues were proposed in therecent literature, the problem is still far from being solved. In this paper,we uncover two factors limiting the VL models' compositional reasoningperformance. These two factors are properties of the paired VL dataset used forfinetuning and pre-training the VL model: (i) the caption quality, or in otherwords `image-alignment', of the texts; and (ii) the `density' of the captionsin the sense of mentioning all the details appearing on the image. We propose afine-tuning approach for automatically treating these factors leveraging astandard VL dataset (CC3M). Applied to CLIP, we demonstrate its significantcompositional reasoning performance increase of up to $\sim27\%$ over the basemodel, up to $\sim20\%$ over the strongest baseline, and by $6.7\%$ on average.

Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models