English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Neural networks have been shown to perform incredibly well in classificationtasks over structured high-dimensional datasets. However, the learning dynamicsof such networks is still poorly understood. In this paper we study in detailthe training dynamics of a simple type of neural network: a single hidden layertrained to perform a classification task. We show that in a suitable mean-fieldlimit this case maps to a single-node learning problem with a time-dependentdataset determined self-consistently from the average nodes population. Wespecialize our theory to the prototypical case of a linearly separable datasetand a linear hinge loss, for which the dynamics can be explicitly solved. Thisallow us to address in a simple setting several phenomena appearing in modernnetworks such as slowing down of training dynamics, crossover between rich andlazy learning, and overfitting. Finally, we asses the limitations of mean-fieldtheory by studying the case of large but finite number of nodes and of trainingsamples.

An analytic theory of shallow networks dynamics for hinge loss classification