English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This paper considers the problem of understanding the behavior of a generalclass of accelerated gradient methods on smooth nonconvex functions. Motivatedby some recent works that have proposed effective algorithms, based on Polyak'sheavy ball method and the Nesterov accelerated gradient method, to achieveconvergence to a local minimum of nonconvex functions, this work proposes abroad class of Nesterov-type accelerated methods and puts forth a rigorousstudy of these methods encompassing the escape from saddle-points andconvergence to local minima through a both asymptotic and a non-asymptoticanalysis. In the asymptotic regime, this paper answers an open question ofwhether Nesterov's accelerated gradient method (NAG) with variable momentumparameter avoids strict saddle points almost surely. This work also developstwo metrics of asymptotic rate of convergence and divergence, and evaluatesthese two metrics for several popular standard accelerated methods such as theNAG, and Nesterov's accelerated gradient with constant momentum (NCM) nearstrict saddle points. In the local regime, this work provides an analysis thatleads to the "linear" exit time estimates from strict saddle neighborhoods fortrajectories of these accelerated methods as well the necessary conditions forthe existence of such trajectories. Finally, this work studies a sub-class ofaccelerated methods that can converge in convex neighborhoods of nonconvexfunctions with a near optimal rate to a local minima and at the same time thissub-class offers superior saddle-escape behavior compared to that of NAG.

Accelerated gradient methods for nonconvex optimization: Escape trajectories from strict saddle points and convergence to local minima