English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Establishing a theoretical analysis that explains why deep learning canoutperform shallow learning such as kernel methods is one of the biggest issuesin the deep learning literature. Towards answering this question, we evaluateexcess risk of a deep learning estimator trained by a noisy gradient descentwith ridge regularization on a mildly overparameterized neural network, anddiscuss its superiority to a class of linear estimators that includes neuraltangent kernel approach, random feature model, other kernel methods, $k$-NNestimator and so on. We consider a teacher-student regression model, andeventually show that any linear estimator can be outperformed by deep learningin a sense of the minimax optimal rate especially for a high dimension setting.The obtained excess bounds are so-called fast learning rate which is fasterthan $O(1/\sqrt{n})$ that is obtained by usual Rademacher complexity analysis.This discrepancy is induced by the non-convex geometry of the model and thenoisy gradient descent used for neural network training provably reaches a nearglobal optimal solution even though the loss landscape is highly non-convex.Although the noisy gradient descent does not employ any explicit or implicitsparsity inducing regularization, it shows a preferable generalizationperformance that dominates linear estimators.

Benefit of deep learning with non-convex noisy gradient descent: Provable excess risk bound and superiority to kernel methods