English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We numerically study the relaxation dynamics of the single layer perceptronwith the spherical constraint. This is the simplest model of neural networksand serves a prototypical mean-field model of both convex and non-convexoptimization problems. The relaxation time of the gradient descent algorithmrapidly increases near the SAT-UNSAT transition point. We numerically confirmthat the first non-zero eigenvalue of the Hessian controls the relaxation time.This first eigenvalue vanishes much faster upon approaching the SAT-UNSATtransition point than the prediction of Marchenko-Pastur law in random matrixtheory derived under the assumption that the set of unsatisfied constraints areuncorrelated. This leads to a non-trivial critical exponent of the relaxationtime in the SAT phase. Using a simple scaling analysis, we show that theisolation of this first eigenvalue from the bulk of spectrum is attributed tothe force balance at the SAT-UNSAT transition point. Finally, we show that theestimated critical exponent of the relaxation time in the non-convex regionagrees very well with that of frictionless spherical particles, which have beenstudied in the context of the jamming transition of granular materials.

Force balance controls the relaxation time of the gradient descent algorithm in the satisfiable phase