English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a new stochastic optimization framework for empirical riskminimization problems such as those that arise in machine learning. Thetraditional approaches, such as (mini-batch) stochastic gradient descent (SGD),utilize an unbiased gradient estimator of the empirical average loss. Incontrast, we develop a computationally efficient method to construct a gradientestimator that is purposely biased toward those observations with highercurrent losses. On the theory side, we show that the proposed method minimizesa new ordered modification of the empirical average loss, and is guaranteed toconverge at a sublinear rate to a global optimum for convex loss and to acritical point for weakly convex (non-convex) loss. Furthermore, we prove a newgeneralization bound for the proposed algorithm. On the empirical side, thenumerical experiments show that our proposed method consistently improves thetest errors compared with the standard mini-batch SGD in various modelsincluding SVM, logistic regression, and deep learning problems.

Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization