English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Motivated by broad applications in machine learning, we study the popularaccelerated stochastic gradient descent (ASGD) algorithm for solving (possiblynonconvex) optimization problems. We characterize the finite-time performanceof this method when the gradients are sampled from Markov processes, and hencebiased and dependent from time step to time step; in contrast, the analysis inexisting work relies heavily on the stochastic gradients being independent andsometimes unbiased. Our main contributions show that under certain (standard)assumptions on the underlying Markov chain generating the gradients, ASGDconverges at the nearly the same rate with Markovian gradient samples as withindependent gradient samples. The only difference is a logarithmic factor thataccounts for the mixing time of the Markov chain. One of the key motivationsfor this study are complicated control problems that can be modeled by a Markovdecision process and solved using reinforcement learning. We apply theaccelerated method to several challenging problems in the OpenAI Gym andMujoco, and show that acceleration can significantly improve the performance ofthe classic temporal difference learning and REINFORCE algorithms.

Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning