English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Generalization properties are a central aspect of the design and analysis oflearning algorithms. One notion that has been considered in many previous worksas leading to good generalization is flat minima, which informally describes aloss surface that is insensitive to noise perturbations. However, the design ofefficient algorithms (that are easy to analyze) to find them is relativelyunder-explored. In this paper, we propose a new algorithm to address thisissue, which minimizes a stochastic optimization objective that averages noiseperturbations injected into the weights of a function. This algorithm is shownto enjoy both theoretical and empirical advantages compared to existingalgorithms involving worst-case perturbations. Theoretically, we show tightconvergence rates of our algorithm to find first-order stationary points of thestochastic objective. Empirically, the algorithm induces a penalty on the traceof the Hessian, leading to iterates that are flatter than SGD and otheralternatives, with tighter generalization gaps. Altogether, this workcontributes a provable and practical algorithm to find flat minima byoptimizing the noise stability properties of a function.

Noise Stability Optimization for Flat Minima with Tight Rates