English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

This paper concerns error bounds for recursive equations subject to Markoviandisturbances. Motivating examples abound within the fields of Markov chainMonte Carlo (MCMC) and Reinforcement Learning (RL), and many of thesealgorithms can be interpreted as special cases of stochastic approximation(SA). It is argued that it is not possible in general to obtain a Hoeffdingbound on the error sequence, even when the underlying Markov chain isreversible and geometrically ergodic, such as the M/M/1 queue. This ismotivation for the focus on mean square error bounds for parameter estimates.It is shown that mean square error achieves the optimal rate of $O(1/n)$,subject to conditions on the step-size sequence. Moreover, the exact constantsin the rate are obtained, which is of great value in algorithm design.

Explicit Mean-Square Error Bounds for Monte-Carlo and Linear Stochastic Approximation