English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Adversarial training has shown its ability in producing models that arerobust to perturbations on the input data, but usually at the expense ofdecrease in the standard accuracy. To mitigate this issue, it is commonlybelieved that more training data will eventually help such adversarially robustmodels generalize better on the benign/unperturbed test data. In this paper,however, we challenge this conventional belief and show that more training datacan hurt the generalization of adversarially robust models in theclassification problems. We first investigate the Gaussian mixtureclassification with a linear loss and identify three regimes based on thestrength of the adversary. In the weak adversary regime, more data improves thegeneralization of adversarially robust models. In the medium adversary regime,with more training data, the generalization loss exhibits a double descentcurve, which implies the existence of an intermediate stage where more trainingdata hurts the generalization. In the strong adversary regime, more data almostimmediately causes the generalization error to increase. Then we move to theanalysis of a two-dimensional classification problem with a 0-1 loss. We provethat more data always hurts the generalization performance of adversariallytrained models with large perturbations. To complement our theoretical results,we conduct empirical studies on Gaussian mixture classification, support vectormachines (SVMs), and linear regression.

The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization