English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The cubic regularized Newton method of Nesterov and Polyak has becomeincreasingly popular for non-convex optimization because of its capability offinding an approximate local solution with second-order guarantee. Severalrecent works extended this method to the setting of minimizing the average of Nsmooth functions by replacing the exact gradients and Hessians with subsampledapproximations. It has been shown that the total Hessian sample complexity canbe reduced to be sublinear in N per iteration by leveraging stochastic variancereduction techniques. We present an adaptive variance reduction scheme forsubsampled Newton method with cubic regularization, and show that the expectedHessian sample complexity is O(N + N^{2/3}\epsilon^{-3/2}) for finding an(\epsilon,\epsilon^{1/2})-approximate local solution (in terms of first andsecond-order guarantees respectively). Moreover, we show that the same Hessiansample complexity retains with fixed sample sizes if exact gradients are used.The techniques of our analysis are different from previous works in that we donot rely on high probability bounds based on matrix concentration inequalities.Instead, we derive and utilize bounds on the 3rd and 4th order moments of theaverage of random matrices, which are of independent interest on their own.

Adaptive Stochastic Variance Reduction for Subsampled Newton Method with Cubic Regularization