English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We present a scalable and effective exploration strategy based on Thompsonsampling for reinforcement learning (RL). One of the key shortcomings ofexisting Thompson sampling algorithms is the need to perform a Gaussianapproximation of the posterior distribution, which is not a good surrogate inmost practical settings. We instead directly sample the Q function from itsposterior distribution, by using Langevin Monte Carlo, an efficient type ofMarkov Chain Monte Carlo (MCMC) method. Our method only needs to perform noisygradient descent updates to learn the exact posterior distribution of the Qfunction, which makes our approach easy to deploy in deep RL. We provide arigorous theoretical analysis for the proposed method and demonstrate that, inthe linear Markov decision process (linear MDP) setting, it has a regret boundof $\tilde{O}(d^{3/2}H^{5/2}\sqrt{T})$, where $d$ is the dimension of thefeature mapping, $H$ is the planning horizon, and $T$ is the total number ofsteps. We apply this approach to deep RL, by using Adam optimizer to performgradient updates. Our approach achieves better or similar results compared withstate-of-the-art deep RL algorithms on several challenging exploration tasksfrom the Atari57 suite.

Provable and Practical: Efficient Exploration in Reinforcement Learning via Langevin Monte Carlo