English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Although Nesterov's accelerated gradient (NAG) methods have been studied fromvarious perspectives, it remains unclear why the most popular forms of NAG musthandle convex and strongly convex objective functions separately. Motivated bythis inconsistency, we propose an NAG method that unifies the existing ones forthe convex and strongly convex cases. We first design a Lagrangian functionthat continuously extends the first Bregman Lagrangian to the strongly convexsetting. As a specific case of the Euler--Lagrange equation for thisLagrangian, we derive an ordinary differential equation (ODE) model, which wecall the unified NAG ODE, that bridges the gap between the ODEs that model NAGfor convex and strongly convex objective functions. We then design the unifiedNAG, a novel momentum method whereby the continuous-time limit corresponds tothe unified ODE. The coefficients and the convergence rates of the unified NAGand unified ODE are continuous in the strong convexity parameter $\mu$ on $[0,+\infty)$. Unlike the existing popular algorithm and ODE for strongly convexobjective functions, the unified NAG and the unified NAG ODE always havesuperior convergence guarantees compared to the known algorithms and ODEs fornon-strongly convex objective functions. This property is beneficial inpractical perspective when considering strongly convex objective functions withsmall $\mu$. Furthermore, we extend our unified dynamics and algorithms to thehigher-order setting. Last but not least, we propose the unified NAG-G ODE, anovel ODE model for minimizing the gradient norm of strongly convex objectivefunctions. Our unified Lagrangian framework is crucial in the process ofconstructing this ODE. Fascinatingly, using our novel tool, called thedifferential kernel, we observe that the unified NAG ODE and the unified NAG-GODE have an anti-transpose relationship.

Unifying Nesterov's Accelerated Gradient Methods for Convex and Strongly  Convex Objective Functions: From Continuous-Time Dynamics to Discrete-Time  Algorithms