English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In 1964, Polyak showed that the Heavy-ball method, the simplest momentumtechnique, accelerates convergence of strongly-convex problems in the vicinityof the solution. While Nesterov later developed a globally accelerated version,Polyak's original algorithm remains simpler and more widely used inapplications such as deep learning. Despite this popularity, the question ofwhether Heavy-ball is also globally accelerated or not has not been fullyanswered yet, and no convincing counterexample has been provided. This islargely due to the difficulty in finding an effective Lyapunov function:indeed, most proofs of Heavy-ball acceleration in the strongly-convex quadraticsetting rely on eigenvalue arguments. Our study adopts a different approach:studying momentum through the lens of quadratic invariants of simple harmonicoscillators. By utilizing the modified Hamiltonian of Stormer-Verletintegrators, we are able to construct a Lyapunov function that demonstrates anO(1/k^2) rate for Heavy-ball in the case of convex quadratic problems. This isa promising first step towards potentially proving the acceleration of Polyak'smomentum method and we hope it inspires further research in this field.

An Accelerated Lyapunov Function for Polyak's Heavy-Ball on Convex Quadratics