English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep neural networks generalize well despite being exceedinglyoverparameterized and being trained without explicit regularization. Thiscurious phenomenon has inspired extensive research activity in establishing itsstatistical principles: Under what conditions is it observed? How do thesedepend on the data and on the training algorithm? When does regularizationbenefit generalization? While such questions remain wide open for deep neuralnets, recent works have attempted gaining insights by studying simpler, oftenlinear, models. Our paper contributes to this growing line of work by examiningbinary linear classification under a generative Gaussian mixture model.Motivated by recent results on the implicit bias of gradient descent, we studyboth max-margin SVM classifiers (corresponding to logistic loss) and min-norminterpolating classifiers (corresponding to least-squares loss). First, weleverage an idea introduced in [V. Muthukumar et al., arXiv:2005.08054, (2020)]to relate the SVM solution to the min-norm interpolating solution. Second, wederive novel non-asymptotic bounds on the classification error of the latter.Combining the two, we present novel sufficient conditions on the covariancespectrum and on the signal-to-noise ratio (SNR) under which interpolatingestimators achieve asymptotically optimal performance as overparameterizationincreases. Interestingly, our results extend to a noisy model with constantprobability noise flips. Contrary to previously studied discriminative datamodels, our results emphasize the crucial role of the SNR and its interplaywith the data covariance. Finally, via a combination of analytical argumentsand numerical demonstrations we identify conditions under which theinterpolating estimator performs better than corresponding regularizedestimates.

Binary Classification of Gaussian Mixtures: Abundance of Support Vectors, Benign Overfitting and Regularization