English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We consider the problem of learning in a non-stationary reinforcementlearning (RL) environment, where the setting can be fully described by apiecewise stationary discrete-time Markov decision process (MDP). We introducea variant of the Restarted Bayesian Online Change-Point Detection algorithm(R-BOCPD) that operates on input streams originating from the more generalmultinomial distribution and provides near-optimal theoretical guarantees interms of false-alarm rate and detection delay. Based on this, we propose animproved version of the UCRL2 algorithm for MDPs with state transition kernelsampled from a multinomial distribution, which we call R-BOCPD-UCRL2. Weperform a finite-time performance analysis and show that R-BOCPD-UCRL2 enjoys afavorable regret bound of $O\left(D O \sqrt{A T K_T \log\left (\frac{T}{\delta}\right) + \frac{K_T \log \frac{K_T}{\delta}}{\min\limits_\ell \:\mathbf{KL}\left({\mathbf{\theta}^{(\ell+1)}}\mid\mid{\mathbf{\theta}^{(\ell)}}\right)}}\right)$,where $D$ is the largest MDP diameter from the set of MDPs defining thepiecewise stationary MDP setting, $O$ is the finite number of states (constantover all changes), $A$ is the finite number of actions (constant over allchanges), $K_T$ is the number of change points up to horizon $T$, and$\mathbf{\theta}^{(\ell)}$ is the transition kernel during the interval$[c_\ell, c_{\ell+1})$, which we assume to be multinomially distributed overthe set of states $\mathbb{O}$. Interestingly, the performance bound does notdirectly scale with the variation in MDP state transition distributions andrewards, ie. can also model abrupt changes. In practice, R-BOCPD-UCRL2outperforms the state-of-the-art in a variety of scenarios in syntheticenvironments. We provide a detailed experimental setup along with a coderepository (upon publication) that can be used to easily reproduce ourexperiments.

Restarted Bayesian Online Change-point Detection for Non-Stationary Markov Decision Processes