English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Maximum likelihood estimation is widely used in training Energy-based models(EBMs). Training requires samples from an unnormalized distribution, which isusually intractable, and in practice, these are obtained by MCMC algorithmssuch as Langevin dynamics. However, since MCMC in high-dimensional spaceconverges extremely slowly, the current understanding of maximum likelihoodtraining, which assumes approximate samples from the model can be drawn, isproblematic. In this paper, we try to understand this training procedure byreplacing Langevin dynamics with deterministic solutions of the associatedgradient descent ODE. Doing so allows us to study the density induced by thedynamics (if the dynamics are invertible), and connect with GANs by treatingthe dynamics as generator models, the initial values as latent variables andthe loss as optimizing a critic defined by the very same energy that determinesthe generator through its gradient. Hence the term - self-adversarial loss. Weshow that reintroducing the noise in the dynamics does not lead to aqualitative change in the behavior, and merely reduces the quality of thegenerator. We thus show that EBM training is effectively a self-adversarialprocedure rather than maximum likelihood estimation.

EBMs Trained with Maximum Likelihood are Generator Models Trained with a Self-adverserial Loss