English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Generalization performance of classifiers in deep learning has recentlybecome a subject of intense study. Deep models, typically over-parametrized,tend to fit the training data exactly. Despite this "overfitting", they performwell on test data, a phenomenon not yet fully understood.  The first point of our paper is that strong performance of overfittedclassifiers is not a unique feature of deep learning. Using six real-world andtwo synthetic datasets, we establish experimentally that kernel machinestrained to have zero classification or near zero regression error perform verywell on test data, even when the labels are corrupted with a high level ofnoise. We proceed to give a lower bound on the norm of zero loss solutions forsmooth kernels, showing that they increase nearly exponentially with data size.We point out that this is difficult to reconcile with the existinggeneralization bounds. Moreover, none of the bounds produce non-trivial resultsfor interpolating solutions.  Second, we show experimentally that (non-smooth) Laplacian kernels easily fitrandom labels, a finding that parallels results for ReLU neural networks. Incontrast, fitting noisy data requires many more epochs for smooth Gaussiankernels. Similar performance of overfitted Laplacian and Gaussian classifierson test, suggests that generalization is tied to the properties of the kernelfunction rather than the optimization process.  Certain key phenomena of deep learning are manifested similarly in kernelmethods in the modern "overfitted" regime. The combination of the experimentaland theoretical results presented in this paper indicates a need for newtheoretical ideas for understanding properties of classical kernel methods. Weargue that progress on understanding deep learning will be difficult until moretractable "shallow" kernel methods are better understood.

To understand deep learning we need to understand kernel learning