English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The stochastic proximal point (SPP) methods have gained recent attention forstochastic optimization, with strong convergence guarantees and superiorrobustness to the classic stochastic gradient descent (SGD) methods showcasedat little to no cost of computational overhead added. In this article, we studya minibatch variant of SPP, namely M-SPP, for solving convex composite riskminimization problems. The core contribution is a set of novel excess riskbounds of M-SPP derived through the lens of algorithmic stability theory.Particularly under smoothness and quadratic growth conditions, we show thatM-SPP with minibatch-size $n$ and iteration count $T$ enjoys an in-expectationfast rate of convergence consisting of an$\mathcal{O}\left(\frac{1}{T^2}\right)$ bias decaying term and an$\mathcal{O}\left(\frac{1}{nT}\right)$ variance decaying term. In thesmall-$n$-large-$T$ setting, this result substantially improves the best knownresults of SPP-type approaches by revealing the impact of noise level of modelon convergence rate. In the complementary small-$T$-large-$n$ regime, weprovide a two-phase extension of M-SPP to achieve comparable convergence rates.Moreover, we derive a near-tight high probability (over the randomness of data)bound on the parameter estimation error of a sampling-without-replacementvariant of M-SPP. Numerical evidences are provided to support our theoreticalpredictions when substantialized to Lasso and logistic regression models.

Sharper Analysis for Minibatch Stochastic Proximal Point Methods: Stability, Smoothness, and Deviation