English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we study the problem of sampling from distributions of theform p(x) \propto e^{-\beta f(x)} for some function f whose values andgradients we can query. This mode of access to f is natural in the scenarios inwhich such problems arise, for instance sampling from posteriors in parametricBayesian models. Classical results show that a natural random walk, Langevindiffusion, mixes rapidly when f is convex. Unfortunately, even in simpleexamples, the applications listed above will entail working with functions fthat are nonconvex -- for which sampling from p may in general require anexponential number of queries.  In this paper, we focus on an aspect of nonconvexity relevant for modernmachine learning applications: existence of invariances (symmetries) in thefunction f, as a result of which the distribution p will have manifolds ofpoints with equal probability. First, we give a recipe for proving mixing timebounds for Langevin diffusion as a function of the geometry of these manifolds.Second, we specialize our arguments to classic matrix factorization-likeBayesian inference problems where we get noisy measurements A(XX^T), X \in R^{d\times k} of a low-rank matrix, i.e. f(X) = \|A(XX^T) - b\|^2_2, X \in R^{d\times k}, and \beta the inverse of the variance of the noise. Such functions fare invariant under orthogonal transformations, and include problems likematrix factorization, sensing, completion. Beyond sampling, Langevin dynamicsis a popular toy model for studying stochastic gradient descent. Along theselines, we believe that our work is an important first step towardsunderstanding how SGD behaves when there is a high degree of symmetry in thespace of parameters the produce the same output.

Fast Convergence for Langevin Diffusion with Manifold Structure