English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper studies the exponential stability of random matrix products drivenby a general (possibly unbounded) state space Markov chain. It is a cornerstonein the analysis of stochastic algorithms in machine learning (e.g. forparameter tracking in online learning or reinforcement learning). The existingresults impose strong conditions such as uniform boundedness of thematrix-valued functions and uniform ergodicity of the Markov chains. Our maincontribution is an exponential stability result for the $p$-th moment of randommatrix product, provided that (i) the underlying Markov chain satisfies asuper-Lyapunov drift condition, (ii) the growth of the matrix-valued functionsis controlled by an appropriately defined function (related to the driftcondition). Using this result, we give finite-time $p$-th moment bounds forconstant and decreasing stepsize linear stochastic approximation schemes withMarkovian noise on general state space. We illustrate these findings for linearvalue-function estimation in reinforcement learning. We provide finite-time$p$-th moment bound for various members of temporal difference (TD) family ofalgorithms.

On the Stability of Random Matrix Product with Markovian Noise: Application to Linear Stochastic Approximation and TD Learning