English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Sharpness-aware minimization (SAM) was proposed to reduce sharpness of minimaand has been shown to enhance generalization performance in various settings.In this work we show that perturbing only the affine normalization parameters(typically comprising 0.1% of the total parameters) in the adversarial step ofSAM can outperform perturbing all of the parameters.This finding generalizes todifferent SAM variants and both ResNet (Batch Normalization) and VisionTransformer (Layer Normalization) architectures. We consider alternative sparseperturbation approaches and find that these do not achieve similar performanceenhancement at such extreme sparsity levels, showing that this behaviour isunique to the normalization layers. Although our findings reaffirm theeffectiveness of SAM in improving generalization performance, they cast doubton whether this is solely caused by reduced sharpness.

Normalization Layers Are All That Sharpness-Aware Minimization Needs