English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Training energy-based models (EBMs) with maximum likelihood estimation onhigh-dimensional data can be both challenging and time-consuming. As a result,there a noticeable gap in sample quality between EBMs and other generativeframeworks like GANs and diffusion models. To close this gap, inspired by therecent efforts of learning EBMs by maximimizing diffusion recovery likelihood(DRL), we propose cooperative diffusion recovery likelihood (CDRL), aneffective approach to tractably learn and sample from a series of EBMs definedon increasingly noisy versons of a dataset, paired with an initializer modelfor each EBM. At each noise level, the initializer model learns to amortize thesampling process of the EBM, and the two models are jointly estimated within acooperative training framework. Samples from the initializer serve as startingpoints that are refined by a few sampling steps from the EBM. With the refinedsamples, the EBM is optimized by maximizing recovery likelihood, while theinitializer is optimized by learning from the difference between the refinedsamples and the initial samples. We develop a new noise schedule and a variancereduction technique to further improve the sample quality. Combining theseadvances, we significantly boost the FID scores compared to existing EBMmethods on CIFAR-10 and ImageNet 32x32, with a 2x speedup over DRL. Inaddition, we extend our method to compositional generation and image inpaintingtasks, and showcase the compatibility of CDRL with classifier-free guidance forconditional generation, achieving similar trade-offs between sample quality andsample diversity as in diffusion models.

Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood