English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent years have seen a growing interest in understanding deep neuralnetworks from an optimization perspective. It is understood now that convergingto low-cost local minima is sufficient for such models to become effective inpractice. However, in this work, we propose a new hypothesis based on recenttheoretical findings and empirical studies that deep neural network modelsactually converge to saddle points with high degeneracy. Our findings from thiswork are new, and can have a significant impact on the development of gradientdescent based methods for training deep networks. We validated our hypothesesusing an extensive experimental evaluation on standard datasets such as MNISTand CIFAR-10, and also showed that recent efforts that attempt to escapesaddles finally converge to saddles with high degeneracy, which we define as`good saddles'. We also verified the famous Wigner's Semicircle Law in ourexperimental results.

Are Saddles Good Enough for Deep Learning?