English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The recently rising markup-to-image generation poses greater challenges ascompared to natural image generation, due to its low tolerance for errors aswell as the complex sequence and context correlations between markup andrendered image. This paper proposes a novel model named "Contrast-augmentedDiffusion Model with Fine-grained Sequence Alignment" (FSA-CDM), whichintroduces contrastive positive/negative samples into the diffusion model toboost performance for markup-to-image generation. Technically, we design afine-grained cross-modal alignment module to well explore the sequencesimilarity between the two modalities for learning robust featurerepresentations. To improve the generalization ability, we propose acontrast-augmented diffusion model to explicitly explore positive and negativesamples by maximizing a novel contrastive variational objective, which ismathematically inferred to provide a tighter bound for the model'soptimization. Moreover, the context-aware cross attention module is developedto capture the contextual information within markup language during thedenoising process, yielding better noise prediction results. Extensiveexperiments are conducted on four benchmark datasets from different domains,and the experimental results demonstrate the effectiveness of the proposedcomponents in FSA-CDM, significantly exceeding state-of-the-art performance byabout 2%-12% DTW improvements. The code will be released athttps://github.com/zgj77/FSACDM.

Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation