English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

It is common practice in deep learning to use overparameterized networks andtrain for as long as possible; there are numerous studies that show, boththeoretically and empirically, that such practices surprisingly do not undulyharm the generalization performance of the classifier. In this paper, weempirically study this phenomenon in the setting of adversarially trained deepnetworks, which are trained to minimize the loss under worst-case adversarialperturbations. We find that overfitting to the training set does in fact harmrobust performance to a very large degree in adversarially robust trainingacross multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) andperturbation models ($\ell_\infty$ and $\ell_2$). Based upon this observedeffect, we show that the performance gains of virtually all recent algorithmicimprovements upon adversarial training can be matched by simply using earlystopping. We also show that effects such as the double descent curve do stilloccur in adversarially trained models, yet fail to explain the observedoverfitting. Finally, we study several classical and modern deep learningremedies for overfitting, including regularization and data augmentation, andfind that no approach in isolation improves significantly upon the gainsachieved by early stopping. All code for reproducing the experiments as well aspretrained model weights and training logs can be found athttps://github.com/locuslab/robust_overfitting.

Overfitting in adversarially robust deep learning