English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Energy-based models (EBMs) are generative models that are usually trained viamaximum likelihood estimation. This approach becomes challenging in genericsituations where the trained energy is non-convex, due to the need to samplethe Gibbs distribution associated with this energy. Using general Fenchelduality results, we derive variational principles dual to maximum likelihoodEBMs with shallow overparametrized neural network energies, both in thefeature-learning and lazy linearized regimes. In the feature-learning regime,this dual formulation justifies using a two time-scale gradient ascent-descent(GDA) training algorithm in which one updates concurrently the particles in thesample space and the neurons in the parameter space of the energy. We alsoconsider a variant of this algorithm in which the particles are sometimesrestarted at random samples drawn from the data set, and show that performingthese restarts at every iteration step corresponds to score matching training.These results are illustrated in simple numerical experiments, which indicatesthat GDA performs best when features and particles are updated using similartime scales.

Dual Training of Energy-Based Models with Overparametrized Shallow Neural Networks