English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Extrapolation is a well-known technique for solving convex optimization andvariational inequalities and recently attracts some attention for non-convexoptimization. Several recent works have empirically shown its success in somemachine learning tasks. However, it has not been analyzed for non-convexminimization and there still remains a gap between the theory and the practice.In this paper, we analyze gradient descent and stochastic gradient descent withextrapolation for finding an approximate first-order stationary point in smoothnon-convex optimization problems. Our convergence upper bounds show that thealgorithms with extrapolation can be accelerated than without extrapolation.

On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization