English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The recently proposed optimization algorithm for deep neural networksSharpness Aware Minimization (SAM) suggests perturbing parameters beforegradient calculation by a gradient ascent step to guide the optimization intoparameter space regions of flat loss. While significant generalizationimprovements and thus reduction of overfitting could be demonstrated, thecomputational costs are doubled due to the additionally needed gradientcalculation, making SAM unfeasible in case of limited computationallycapacities. Motivated by Nesterov Accelerated Gradient (NAG) we proposeMomentum-SAM (MSAM), which perturbs parameters in the direction of theaccumulated momentum vector to achieve low sharpness without significantcomputational overhead or memory demands over SGD or Adam. We evaluate MSAM indetail and reveal insights on separable mechanisms of NAG, SAM and MSAMregarding training optimization and generalization. Code is available athttps://github.com/MarlonBecker/MSAM.

Momentum-SAM: Sharpness Aware Minimization without Computational Overhead