English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The field of text-conditioned image generation has made unparalleled progresswith the recent advent of latent diffusion models. While remarkable, as thecomplexity of given text input increases, the state-of-the-art diffusion modelsmay still fail in generating images which accurately convey the semantics ofthe given prompt. Furthermore, it has been observed that such misalignments areoften left undetected by pretrained multi-modal models such as CLIP. To addressthese problems, in this paper we explore a simple yet effective decompositionalapproach towards both evaluation and improvement of text-to-image alignment. Inparticular, we first introduce a Decompositional-Alignment-Score which given acomplex prompt decomposes it into a set of disjoint assertions. The alignmentof each assertion with generated images is then measured using a VQA model.Finally, alignment scores for different assertions are combined aposteriori togive the final text-to-image alignment score. Experimental analysis revealsthat the proposed alignment metric shows significantly higher correlation withhuman ratings as opposed to traditional CLIP, BLIP scores. Furthermore, we alsofind that the assertion level alignment scores provide a useful feedback whichcan then be used in a simple iterative procedure to gradually increase theexpression of different assertions in the final image outputs. Human userstudies indicate that the proposed approach surpasses previous state-of-the-artby 8.7% in overall text-to-image alignment accuracy. Project page for our paperis available at https://1jsingh.github.io/divide-evaluate-and-refine

Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback