English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Latent image representations arising from vision-language models have provedimmensely useful for a variety of downstream tasks. However, their utility islimited by their entanglement with respect to different visual attributes. Forinstance, recent work has shown that CLIP image representations are oftenbiased toward specific visual properties (such as objects or actions) in anunpredictable manner. In this paper, we propose to separate representations ofthe different visual modalities in CLIP's joint vision-language space byleveraging the association between parts of speech and specific visual modes ofvariation (e.g. nouns relate to objects, adjectives describe appearance). Thisis achieved by formulating an appropriate component analysis model that learnssubspaces capturing variability corresponding to a specific part of speech,while jointly minimising variability to the rest. Such a subspace yieldsdisentangled representations of the different visual properties of an image ortext in closed form while respecting the underlying geometry of the manifold onwhich the representations lie. What's more, we show the proposed modeladditionally facilitates learning subspaces corresponding to specific visualappearances (e.g. artists' painting styles), which enables the selectiveremoval of entire visual themes from CLIP-based text-to-image synthesis. Wevalidate the model both qualitatively, by visualising the subspace projectionswith a text-to-image model and by preventing the imitation of artists' styles,and quantitatively, through class invariance metrics and improvements tobaseline zero-shot classification.

Parts of Speech-Grounded Subspaces in Vision-Language Models