English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Score matching is an approach to learning probability distributionsparametrized up to a constant of proportionality (e.g. Energy-Based Models).The idea is to fit the score of the distribution, rather than the likelihood,thus avoiding the need to evaluate the constant of proportionality. Whilethere's a clear algorithmic benefit, the statistical "cost'' can be steep:recent work by Koehler et al. 2022 showed that for distributions that have poorisoperimetric properties (a large Poincar\'e or log-Sobolev constant), scorematching is substantially statistically less efficient than maximum likelihood.However, many natural realistic distributions, e.g. multimodal distributions assimple as a mixture of two Gaussians in one dimension -- have a poor Poincar\'econstant.  In this paper, we show a close connection between the mixing time of a broadclass of Markov processes with generator $\mathcal{L}$ and an appropriatelychosen generalized score matching loss that tries to fit $\frac{\mathcal{O}p}{p}$. This allows us to adapt techniques to speed up Markov chains toconstruct better score-matching losses. In particular, ``preconditioning'' thediffusion can be translated to an appropriate ``preconditioning'' of the scoreloss. Lifting the chain by adding a temperature like in simulated tempering canbe shown to result in a Gaussian-convolution annealed score matching loss,similar to Song and Ermon, 2019. Moreover, we show that if the distributionbeing learned is a finite mixture of Gaussians in $d$ dimensions with a sharedcovariance, the sample complexity of annealed score matching is polynomial inthe ambient dimension, the diameter of the means, and the smallest and largesteigenvalues of the covariance -- obviating the Poincar\'e constant-based lowerbounds of the basic score matching loss shown in Koehler et al. 2022.

Fit Like You Sample: Sample-Efficient Generalized Score Matching from Fast Mixing Diffusions