English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

For linear classifiers, the relationship between (normalized) output marginand generalization is captured in a clear and simple bound -- a large outputmargin implies good generalization. Unfortunately, for deep models, thisrelationship is less clear: existing analyses of the output margin givecomplicated bounds which sometimes depend exponentially on depth. In this work,we propose to instead analyze a new notion of margin, which we call the"all-layer margin." Our analysis reveals that the all-layer margin has a clearand direct relationship with generalization for deep models. This enables thefollowing concrete applications of the all-layer margin: 1) by analyzing theall-layer margin, we obtain tighter generalization bounds for neural nets whichdepend on Jacobian and hidden layer norms and remove the exponential dependencyon depth 2) our neural net results easily translate to the adversarially robustsetting, giving the first direct analysis of robust test error for deepnetworks, and 3) we present a theoretically inspired training algorithm forincreasing the all-layer margin. Our algorithm improves both clean andadversarially robust test performance over strong baselines in practice.

Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin