English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Adversarial training is a widely used method to improve the robustness ofdeep neural networks (DNNs) over adversarial perturbations. However, it isempirically observed that adversarial training on over-parameterized networksoften suffers from the \textit{robust overfitting}: it can achieve almost zeroadversarial training error while the robust generalization performance is notpromising. In this paper, we provide a theoretical understanding of thequestion of whether overfitted DNNs in adversarial training can generalize froman approximation viewpoint. Specifically, our main results are summarized intothree folds: i) For classification, we prove by construction the existence ofinfinitely many adversarial training classifiers on over-parameterized DNNsthat obtain arbitrarily small adversarial training error (overfitting), whereasachieving good robust generalization error under certain conditions concerningthe data quality, well separated, and perturbation level. ii) Linearover-parameterization (meaning that the number of parameters is only slightlylarger than the sample size) is enough to ensure such existence if the targetfunction is smooth enough. iii) For regression, our results demonstrate thatthere also exist infinitely many overfitted DNNs with linearover-parameterization in adversarial training that can achieve almost optimalrates of convergence for the standard generalization error. Overall, ouranalysis points out that robust overfitting can be avoided but the requiredmodel capacity will depend on the smoothness of the target function, while arobust generalization gap is inevitable. We hope our analysis will give abetter understanding of the mathematical foundations of robustness in DNNs froman approximation view.

Can overfitted deep neural networks in adversarial training generalize? -- An approximation viewpoint