English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Contrastively trained vision-language models have achieved remarkableprogress in vision and language representation learning, leading tostate-of-the-art models for various downstream multimodal tasks. However,recent research has highlighted severe limitations of these models in theirability to perform compositional reasoning over objects, attributes, andrelations. Scene graphs have emerged as an effective way to understand imagescompositionally. These are graph-structured semantic representations of imagesthat contain objects, their attributes, and relations with other objects in ascene. In this work, we consider the scene graph parsed from text as a proxyfor the image scene graph and propose a graph decomposition and augmentationframework along with a coarse-to-fine contrastive learning objective betweenimages and text that aligns sentences of various complexities to the sameimage. Along with this, we propose novel negative mining techniques in thescene graph space for improving attribute binding and relation understanding.Through extensive experiments, we demonstrate the effectiveness of our approachthat significantly improves attribute binding, relation understanding,systematic generalization, and productivity on multiple recently proposedbenchmarks (For example, improvements upto $18\%$ for systematicgeneralization, $16.5\%$ for relation understanding over a strong baseline),while achieving similar or better performance than CLIP on various generalmultimodal tasks.

Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality