English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In this paper, we study the dynamics of gradient descent in learning neuralnetworks for classification problems. Unlike in existing works, we consider thelinearly non-separable case where the training data of different classes lie inorthogonal subspaces. We show that when the network has sufficient (but notexceedingly large) number of neurons, (1) the corresponding minimizationproblem has a desirable landscape where all critical points are global minimawith perfect classification; (2) gradient descent is guaranteed to converge tothe global minima. Moreover, we discovered a geometric condition on the networkweights so that when it is satisfied, the weight evolution transitions from aslow phase of weight direction spreading to a fast phase of weight convergence.The geometric condition says that the convex hull of the weights projected onthe unit sphere contains the origin.

Global Convergence and Geometric Characterization of Slow to Fast Weight  Evolution in Neural Network Training for Classifying Linearly Non-Separable  Data