English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

It has been recognized that a heavily overparameterized artificial neuralnetwork exhibits surprisingly good generalization performance in variousmachine-learning tasks. Recent theoretical studies have made attempts to unveilthe mystery of the overparameterization. In most of those previous works, theoverparameterization is achieved by increasing the width of the network, whilethe effect of increasing the depth has remained less well understood. In thiswork, we investigate the effect of increasing the depth within anoverparameterized regime. To gain an insight into the advantage of depth, weintroduce local and global labels as abstract but simple classification rules.It turns out that the locality of the relevant feature for a givenclassification rule plays a key role; our experimental results suggest thatdeeper is better for local labels, whereas shallower is better for globallabels. We also compare the results of finite networks with those of the neuraltangent kernel (NTK), which is equivalent to an infinitely wide network with aproper initialization and an infinitesimal learning rate. It is shown that theNTK does not correctly capture the depth dependence of the generalizationperformance, which indicates the importance of the feature learning rather thanthe lazy learning.

Is deeper better? It depends on locality of relevant features