English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Existing autoregressive models follow the two-stage generation paradigm thatfirst learns a codebook in the latent space for image reconstruction and thencompletes the image generation autoregressively based on the learned codebook.However, existing codebook learning simply models all local region informationof images without distinguishing their different perceptual importance, whichbrings redundancy in the learned codebook that not only limits the next stage'sautoregressive model's ability to model important structure but also results inhigh training cost and slow generation speed. In this study, we borrow the ideaof importance perception from classical image coding theory and propose a noveltwo-stage framework, which consists of Masked Quantization VAE (MQ-VAE) andStackformer, to relieve the model from modeling redundancy. Specifically,MQ-VAE incorporates an adaptive mask module for masking redundant regionfeatures before quantization and an adaptive de-mask module for recovering theoriginal grid image feature map to faithfully reconstruct the original imagesafter quantization. Then, Stackformer learns to predict the combination of thenext code and its position in the feature map. Comprehensive experiments onvarious image generation validate our effectiveness and efficiency. Code willbe released at https://github.com/CrossmodalGroup/MaskedVectorQuantization.

Not All Image Regions Matter: Masked Vector Quantization for Autoregressive Image Generation