English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

First-order methods for stochastic optimization have undeniable relevance, inpart due to their pivotal role in machine learning. Variance reduction forthese algorithms has become an important research topic. In contrast to commonapproaches, which rarely leverage global models of the objective function, weexploit convexity and L-smoothness to improve the noisy estimates outputted bythe stochastic gradient oracle. Our method, named COCO denoiser, is the jointmaximum likelihood estimator of multiple function gradients from their noisyobservations, subject to co-coercivity constraints between them. The resultingestimate is the solution of a convex Quadratically Constrained QuadraticProblem. Although this problem is expensive to solve by interior point methods,we exploit its structure to apply an accelerated first-order algorithm, theFast Dual Proximal Gradient method. Besides analytically characterizing theproposed estimator, we show empirically that increasing the number andproximity of the queried points leads to better gradient estimates. We alsoapply COCO in stochastic settings by plugging it in existing algorithms, suchas SGD, Adam or STRSAGA, outperforming their vanilla versions, even inscenarios where our modelling assumptions are mismatched.

COCO Denoiser: Using Co-Coercivity for Variance Reduction in Stochastic Convex Optimization