English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Inspired by recent advances in diffusion models, which are reminiscent ofdenoising autoencoders, we investigate whether they can acquire discriminativerepresentations for classification via generative pre-training. This papershows that the networks in diffusion models, namely denoising diffusionautoencoders (DDAE), are unified self-supervised learners: by pre-training onunconditional image generation, DDAE has already learned stronglylinear-separable representations within its intermediate layers withoutauxiliary encoders, thus making diffusion pre-training emerge as a generalapproach for generative-and-discriminative dual learning. To validate this, weconduct linear probe and fine-tuning evaluations. Our diffusion-based approachachieves 95.9% and 50.0% linear evaluation accuracies on CIFAR-10 andTiny-ImageNet, respectively, and is comparable to contrastive learning andmasked autoencoders for the first time. Transfer learning from ImageNet alsoconfirms the suitability of DDAE for Vision Transformers, suggesting thepotential to scale DDAEs as unified foundation models. Code is available atgithub.com/FutureXiang/ddae.

Denoising Diffusion Autoencoders are Unified Self-supervised Learners