English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

To achieve near-zero training error in a classification problem, the layersof a deep network have to disentangle the manifolds of data points withdifferent labels, to facilitate the discrimination. However, excessive classseparation can bring to overfitting since good generalisation requires learninginvariant features, which involve some level of entanglement. We report onnumerical experiments showing how the optimisation dynamics findsrepresentations that balance these opposing tendencies with a non-monotonictrend. After a fast segregation phase, a slower rearrangement (conserved acrossdata sets and architectures) increases the class entanglement. The trainingerror at the inversion is remarkably stable under subsampling, and acrossnetwork initialisations and optimisers, which characterises it as a propertysolely of the data structure and (very weakly) of the architecture. Theinversion is the manifestation of tradeoffs elicited by well-defined andmaximally stable elements of the training set, coined "stragglers",particularly influential for generalisation.

Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation