English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Gradient descent (GD) and stochastic gradient descent (SGD) are theworkhorses of large-scale machine learning. While classical theory focused onanalyzing the performance of these methods in convex optimization problems, themost notable successes in machine learning have involved nonconvexoptimization, and a gap has arisen between theory and practice. Indeed,traditional analyses of GD and SGD show that both algorithms converge tostationary points efficiently. But these analyses do not take into account thepossibility of converging to saddle points. More recent theory has shown thatGD and SGD can avoid saddle points, but the dependence on dimension in theseanalyses is polynomial. For modern machine learning, where the dimension can bein the millions, such dependence would be catastrophic. We analyze perturbedversions of GD and SGD and show that they are truly efficient---their dimensiondependence is only polylogarithmic. Indeed, these algorithms converge tosecond-order stationary points in essentially the same time as they take toconverge to classical first-order stationary points.

On Nonconvex Optimization for Machine Learning: Gradients, Stochasticity, and Saddle Points