English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we propose a new technique named \textit{StochasticPath-Integrated Differential EstimatoR} (SPIDER), which can be used to trackmany deterministic quantities of interest with significantly reducedcomputational cost. We apply SPIDER to two tasks, namely the stochasticfirst-order and zeroth-order methods. For stochastic first-order method,combining SPIDER with normalized gradient descent, we propose two newalgorithms, namely SPIDER-SFO and SPIDER-SFO\textsuperscript{+}, that solvenon-convex stochastic optimization problems using stochastic gradients only. Weprovide sharp error-bound results on their convergence rates. In special, weprove that the SPIDER-SFO and SPIDER-SFO\textsuperscript{+} algorithms achievea record-breaking gradient computation cost of $\mathcal{O}\left( \min( n^{1/2}\epsilon^{-2}, \epsilon^{-3} ) \right)$ for finding an $\epsilon$-approximatefirst-order and $\tilde{\mathcal{O}}\left( \min( n^{1/2}\epsilon^{-2}+\epsilon^{-2.5}, \epsilon^{-3} ) \right)$ for finding an$(\epsilon, \mathcal{O}(\epsilon^{0.5}))$-approximate second-order stationarypoint, respectively. In addition, we prove that SPIDER-SFO nearly matches thealgorithmic lower bound for finding approximate first-order stationary pointsunder the gradient Lipschitz assumption in the finite-sum setting. Forstochastic zeroth-order method, we prove a cost of $\mathcal{O}( d \min(n^{1/2} \epsilon^{-2}, \epsilon^{-3}) )$ which outperforms all existingresults.

SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator