English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Existing analysis of AdaGrad and other adaptive methods for smooth convexoptimization is typically for functions with bounded domain diameter. Inunconstrained problems, previous works guarantee an asymptotic convergence ratewithout an explicit constant factor that holds true for the entire functionclass. Furthermore, in the stochastic setting, only a modified version ofAdaGrad, different from the one commonly used in practice, in which the latestgradient is not used to update the stepsize, has been analyzed. Our paper aimsat bridging these gaps and developing a deeper understanding of AdaGrad and itsvariants in the standard setting of smooth convex functions as well as the moregeneral setting of quasar convex functions. First, we demonstrate newtechniques to explicitly bound the convergence rate of the vanilla AdaGrad forunconstrained problems in both deterministic and stochastic settings. Second,we propose a variant of AdaGrad for which we can show the convergence of thelast iterate, instead of the average iterate. Finally, we give new acceleratedadaptive algorithms and their convergence guarantee in the deterministicsetting with explicit dependency on the problem parameters, improving upon theasymptotic rate shown in previous works.

On the Convergence of AdaGrad(Norm) on $\R^{d}$: Beyond Convexity, Non-Asymptotic Rate and Acceleration