English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Among attempts at giving a theoretical account of the success of deep neuralnetworks, a recent line of work has identified a so-called lazy training regimein which the network can be well approximated by its linearization aroundinitialization. Here we investigate the comparative effect of the lazy (linear)and feature learning (non-linear) regimes on subgroups of examples based ontheir difficulty. Specifically, we show that easier examples are given moreweight in feature learning mode, resulting in faster training compared to moredifficult ones. In other words, the non-linear dynamics tends to sequentializethe learning of examples of increasing difficulty. We illustrate thisphenomenon across different ways to quantify example difficulty, includingc-score, label noise, and in the presence of easy-to-learn spuriouscorrelations. Our results reveal a new understanding of how deep networksprioritize resources across example difficulty.

Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty