English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large-scale non-convex optimization problems are expensive to solve due tocomputational and memory costs. To reduce the costs, first-order(computationally efficient) and asynchronous-parallel (memory efficient)algorithms are necessary to minimize non-convex functions in machine learning.However, asynchronous-first-order methods applied within non-convex settingsrun into two difficulties: (i) parallelization delays, which affect convergenceby disrupting the monotonicity of first-order methods, and (ii) sub-optimalsaddle points where the gradient is zero. To solve these two difficulties, wepropose an asynchronous-coordinate-gradient-descent algorithm shown to convergeto local minima with a bounded delay. Our algorithm overcomesparallelization-delay issues by using a carefully constructed Hamiltonianfunction. We prove that our designed kinetic-energy term, incorporated withinthe Hamiltonian, allows our algorithm to decrease monotonically per iteration.Next, our algorithm steers iterates clear of saddle points by utilizing aperturbation sub-routine. Similar to other state-of-the-art (SOTA) algorithms,we achieve a poly-logarithmic convergence rate with respect to dimension.Unlike other SOTA algorithms, which are synchronous, our work is the first tostudy how parallelization delays affect the convergence rate of asynchronousfirst-order algorithms. We prove that our algorithm outperforms synchronouscounterparts under large parallelization delays, with convergence dependingsublinearly with respect to delays. To our knowledge, this is the first localoptima convergence result of a first-order asynchronous algorithm fornon-convex settings.

Escaping From Saddle Points Using Asynchronous Coordinate Gradient Descent