English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider the problem of constraining diffusion model outputs with auser-supplied reference image. Our key objective is to extract multipleattributes (e.g., color, object, layout, style) from this single referenceimage, and then generate new samples with them. One line of existing workproposes to invert the reference images into a single textual conditioningvector, enabling generation of new samples with this learned token. Thesemethods, however, do not learn multiple tokens that are necessary to conditionmodel outputs on the multiple attributes noted above. Another line oftechniques expand the inversion space to learn multiple embeddings but they dothis only along the layer dimension (e.g., one per layer of the DDPM model) orthe timestep dimension (one for a set of timesteps in the denoising process),leading to suboptimal attribute disentanglement. To address the aforementionedgaps, the first contribution of this paper is an extensive analysis todetermine which attributes are captured in which dimension of the denoisingprocess. As noted above, we consider both the time-step dimension (in reversedenoising) as well as the DDPM model layer dimension. We observe that often asubset of these attributes are captured in the same set of model layers and/oracross same denoising timesteps. For instance, color and style are capturedacross same U-Net layers, whereas layout and color are captured across sametimestep stages. Consequently, an inversion process that is designed only forthe time-step dimension or the layer dimension is insufficient to disentangleall attributes. This leads to our second contribution where we design a newmulti-attribute inversion algorithm, MATTE, with associateddisentanglement-enhancing regularization losses, that operates across bothdimensions and explicitly leads to four disentangled tokens (color, style,layout, and object).

An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesis