English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the pursuit of explaining implicit regularization in deep learning,prominent focus was given to matrix and tensor factorizations, which correspondto simplified neural networks. It was shown that these models exhibit animplicit tendency towards low matrix and tensor ranks, respectively. Drawingcloser to practical deep learning, the current paper theoretically analyzes theimplicit regularization in hierarchical tensor factorization, a modelequivalent to certain deep convolutional neural networks. Through a dynamicalsystems lens, we overcome challenges associated with hierarchy, and establishimplicit regularization towards low hierarchical tensor rank. This translatesto an implicit regularization towards locality for the associated convolutionalnetworks. Inspired by our theory, we design explicit regularizationdiscouraging locality, and demonstrate its ability to improve the performanceof modern convolutional networks on non-local tasks, in defiance ofconventional wisdom by which architectural changes are needed. Our workhighlights the potential of enhancing neural networks via theoretical analysisof their implicit regularization.

Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks