English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We introduce a new theoretical framework to analyze deep learningoptimization with connection to its generalization error. Existing frameworkssuch as mean field theory and neural tangent kernel theory for neural networkoptimization analysis typically require taking limit of infinite width of thenetwork to show its global convergence. This potentially makes it difficult todirectly deal with finite width network; especially in the neural tangentkernel regime, we cannot reveal favorable properties of neural networks beyondkernel methods. To realize more natural analysis, we consider a completelydifferent approach in which we formulate the parameter training as atransportation map estimation and show its global convergence via the theory ofthe infinite dimensional Langevin dynamics. This enables us to analyze narrowand wide networks in a unifying manner. Moreover, we give generalization gapand excess risk bounds for the solution obtained by the dynamics. The excessrisk bound achieves the so-called fast learning rate. In particular, we show anexponential convergence for a classification problem and a minimax optimal ratefor a regression problem.

Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics