English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the detailed path-wise behavior of the discrete-time Langevinalgorithm for non-convex Empirical Risk Minimization (ERM) through the lens ofmetastability, adopting some techniques from Berglund and Gentz (2003.  For a particular local optimum of the empirical risk, with an arbitraryinitialization, we show that, with high probability, at least one of thefollowing two events will occur: (1) the Langevin trajectory ends up somewhereoutside the $\varepsilon$-neighborhood of this particular optimum within ashort recurrence time; (2) it enters this $\varepsilon$-neighborhood by therecurrence time and stays there until a potentially exponentially long escapetime. We call this phenomenon empirical metastability.  This two-timescale characterization aligns nicely with the existingliterature in the following two senses. First, the effective recurrence time(i.e., number of iterations multiplied by stepsize) is dimension-independent,and resembles the convergence time of continuous-time deterministic GradientDescent (GD). However unlike GD, the Langevin algorithm does not require strongconditions on local initialization, and has the possibility of eventuallyvisiting all optima. Second, the scaling of the escape time is consistent withthe Eyring-Kramers law, which states that the Langevin scheme will eventuallyvisit all local minima, but it will take an exponentially long time to transitamong them. We apply this path-wise concentration result in the context ofstatistical learning to examine local notions of generalization and optimality.

Local Optimality and Generalization Guarantees for the Langevin Algorithm via Empirical Metastability