English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper concerns quasi-stochastic approximation (QSA) to solve rootfinding problems commonly found in applications to optimization andreinforcement learning. The general constant gain algorithm may be expressed asthe time-inhomogeneous ODE $ \frac{d}{dt} \Theta_t= \alpha f_t (\Theta_t)$,with state process $\Theta$ evolving on $\mathbb{R}^n$. Theory is based on analmost periodic vector field, so that in particular the time average of$f_t(\theta)$ defines the time-homogeneous mean vector field $\bar{f} \colon\mathbb{R}^d \to \mathbb{R}^d$ with $\bar{f}(\theta^*)=0$. Under smoothnessassumptions on the functions involved, the following exact representation isobtained: \[  \frac{d}{dt} \Theta_t= \alpha [\bar{f}(\Theta_t) -\alpha \bar \Upsilon_t +\alpha^2 \mathcal{W}_t^0 + \alpha \frac{d}{dt} \mathcal{W}_t^1 +\frac{d^2}{dt^2} \mathcal{W}_t^2] \] along with formulae for the smooth signals$\{\bar \Upsilon_t , \mathcal{W}_t^i : i=0, 1, 2\}$. This new representation,combined with new conditions for ultimate boundedness, has many applicationsfor furthering the theory of QSA and its applications, including the followingimplications developed in this paper:  (i) A proof that the estimation error $\| \Theta_t - \theta^* \|$ is of order$O(\alpha)$, but can be reduced to $O(\alpha^2)$ using a second order linearfilter.  (ii) In application to extremum seeking control, it is found that the resultsdo not apply because the standard algorithms are not Lipschitz continuous. Anew approach is presented to ensure that the required Lipschitz bounds hold,and from this we obtain stability, transient bounds, and asymptotic bias oforder $O(\alpha^2)$, and asymptotic variance of order $O(\alpha^4)$.  (iii) It is in general possible to obtain better than $O(\alpha)$ bounds onerror in traditional stochastic approximation when there is Markovian noise.

Markovian Foundations for Quasi-Stochastic Approximation with Applications to Extremum Seeking Control