English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The paper concerns the $d$-dimensional stochastic approximation recursion, $$\theta_{n+1}= \theta_n + \alpha_{n + 1} f(\theta_n, \Phi_{n+1}) $$ in which$\Phi$ is a geometrically ergodic Markov chain on a general state space$\textsf{X}$ with stationary distribution $\pi$, and$f:\Re^d\times\textsf{X}\to\Re^d$.  The main results are established under a version of the Donsker-VaradhanLyapunov drift condition known as (DV3), and a stability condition for the meanflow with vector field $\bar{f}(\theta)=\textsf{E}[f(\theta,\Phi)]$, with$\Phi\sim\pi$.  (i) $\{ \theta_n\}$ is convergent a.s. and in $L_4$ to the unique root$\theta^*$ of $\bar{f}(\theta)$.  (ii) A functional CLT is established, as well as the usual one-dimensionalCLT for the normalized error.  (iii) The CLT holds for the normalized version, $z_n{=:} \sqrt{n}(\theta^{\text{PR}}_n -\theta^*)$, of the averaged parameters,$\theta^{\text{PR}}_n {=:} n^{-1} \sum_{k=1}^n\theta_k$, subject to standardassumptions on the step-size. Moreover, the normalized covariance converges, $$\lim_{n \to \infty} n \textsf{E} [ {\widetilde{\theta}}^{\text{ PR}}_n({\widetilde{\theta}}^{\text{ PR}}_n)^T ] = \Sigma_\theta^*,\;\;\;\textit{with$\widetilde{\theta}^{\text{ PR}}_n = \theta^{\text{ PR}}_n -\theta^*$,}  $$ where $\Sigma_\theta^*$ is the minimal covariance of Polyak and Ruppert.  (iv) An example is given where $f$ and $\bar{f}$ are linear in $\theta$, andthe Markov chain $\Phi$ is geometrically ergodic but does not satisfy (DV3).While the algorithm is convergent, the second moment is unbounded: $ \textsf{E}[ \| \theta_n \|^2 ] \to \infty$ as $n\to\infty$.

The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning