English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider Sharpness-Aware Minimization (SAM), a gradient-based optimizationmethod for deep networks that has exhibited performance improvements on imageand language prediction problems. We show that when SAM is applied with aconvex quadratic objective, for most random initializations it converges to acycle that oscillates between either side of the minimum in the direction withthe largest curvature, and we provide bounds on the rate of convergence.  In the non-quadratic case, we show that such oscillations effectively performgradient descent, with a smaller step-size, on the spectral norm of theHessian. In such cases, SAM's update may be regarded as a third derivative --the derivative of the Hessian in the leading eigenvector direction -- thatencourages drift toward wider minima.

The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima