English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper introduces PROMISE ($\textbf{Pr}$econditioned Stochastic$\textbf{O}$ptimization $\textbf{M}$ethods by $\textbf{I}$ncorporating$\textbf{S}$calable Curvature $\textbf{E}$stimates), a suite of sketching-basedpreconditioned stochastic gradient algorithms for solving large-scale convexoptimization problems arising in machine learning. PROMISE includespreconditioned versions of SVRG, SAGA, and Katyusha; each algorithm comes witha strong theoretical analysis and effective default hyperparameter values. Incontrast, traditional stochastic gradient methods require carefulhyperparameter tuning to succeed, and degrade in the presence ofill-conditioning, a ubiquitous phenomenon in machine learning. Empirically, weverify the superiority of the proposed algorithms by showing that, usingdefault hyperparameter values, they outperform or match popular tunedstochastic gradient optimizers on a test bed of $51$ ridge and logisticregression problems assembled from benchmark machine learning repositories. Onthe theoretical side, this paper introduces the notion of quadratic regularityin order to establish linear convergence of all proposed methods even when thepreconditioner is updated infrequently. The speed of linear convergence isdetermined by the quadratic regularity ratio, which often provides a tighterbound on the convergence rate compared to the condition number, both in theoryand in practice, and explains the fast global linear convergence of theproposed methods.

PROMISE: Preconditioned Stochastic Optimization Methods by Incorporating Scalable Curvature Estimates