English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Although deep generative models have gained a lot of attention, most of theexisting works are designed for unimodal generation. In this paper, we explorea new method for unconditional image-text pair generation. We design MultimodalCross-Quantization VAE (MXQ-VAE), a novel vector quantizer for joint image-textrepresentations, with which we discover that a joint image-text representationspace is effective for semantically consistent image-text pair generation. Tolearn a multimodal semantic correlation in a quantized space, we combine VQ-VAEwith a Transformer encoder and apply an input masking strategy. Specifically,MXQ-VAE accepts a masked image-text pair as input and learns a quantized jointrepresentation space, so that the input can be converted to a unified codesequence, then we perform unconditional image-text pair generation with thecode sequence. Extensive experiments show the correlation between the quantizedjoint space and the multimodal generation capability on synthetic andreal-world datasets. In addition, we demonstrate the superiority of ourapproach in these two aspects over several baselines. The source code ispublicly available at: https://github.com/ttumyche/MXQ-VAE.

Unconditional Image-Text Pair Generation with Multimodal Cross Quantizer