English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Most existing theoretical analysis of reinforcement learning (RL) is limitedto the tabular setting or linear models due to the difficulty in dealing withfunction approximation in high dimensional space with an uncertain environment.This work offers a fresh perspective into this challenge by analyzing RL in ageneral reproducing kernel Hilbert space (RKHS). We consider a family of Markovdecision processes $\mathcal{M}$ of which the reward functions lie in the unitball of an RKHS and transition probabilities lie in a given arbitrary set. Wedefine a quantity called perturbational complexity by distribution mismatch$\Delta_{\mathcal{M}}(\epsilon)$ to characterize the complexity of theadmissible state-action distribution space in response to a perturbation in theRKHS with scale $\epsilon$. We show that $\Delta_{\mathcal{M}}(\epsilon)$ givesboth the lower bound of the error of all possible algorithms and the upperbound of two specific algorithms (fitted reward and fitted Q-iteration) for theRL problem. Hence, the decay of $\Delta_\mathcal{M}(\epsilon)$ with respect to$\epsilon$ measures the difficulty of the RL problem on $\mathcal{M}$. Wefurther provide some concrete examples and discuss whether$\Delta_{\mathcal{M}}(\epsilon)$ decays fast or not in these examples. As abyproduct, we show that when the reward functions lie in a high dimensionalRKHS, even if the transition probability is known and the action space isfinite, it is still possible for RL problems to suffer from the curse ofdimensionality.

Perturbational Complexity by Distribution Mismatch: A Systematic Analysis of Reinforcement Learning in Reproducing Kernel Hilbert Space