English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Motivated by the super-diffusivity of self-repelling random walk, which hasroots in statistical physics, this paper develops a new perturbation mechanismfor optimization algorithms. In this mechanism, perturbations are adapted tothe history of states via the notion of occupation time. After integrating thismechanism into the framework of perturbed gradient descent (PGD) and perturbedaccelerated gradient descent (PAGD), two new algorithms are proposed: perturbedgradient descent adapted to occupation time (PGDOT) and its accelerated version(PAGDOT). PGDOT and PAGDOT are shown to converge to second-order stationarypoints at least as fast as PGD and PAGD, respectively, and thus they areguaranteed to avoid getting stuck at non-degenerate saddle points. Thetheoretical analysis is corroborated by empirical studies in which the newalgorithms consistently escape saddle points and outperform not only theircounterparts, PGD and PAGD, but also other popular alternatives includingstochastic gradient descent, Adam, AMSGrad, and RMSProp.

Escaping Saddle Points Efficiently with Occupation-Time-Adapted Perturbations