English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The classical theory of reinforcement learning (RL) has focused on tabularand linear representations of value functions. Further progress hinges oncombining RL with modern function approximators such as kernel functions anddeep neural networks, and indeed there have been many empirical successes thathave exploited such combinations in large-scale applications. There areprofound challenges, however, in developing a theory to support thisenterprise, most notably the need to take into consideration theexploration-exploitation tradeoff at the core of RL in conjunction with thecomputational and statistical tradeoffs that arise in modernfunction-approximation-based learning systems. We approach these challenges bystudying an optimistic modification of the least-squares value iterationalgorithm, in the context of the action-value function  represented by a kernel function or an overparameterized neural network. Weestablish both polynomial runtime complexity and polynomial sample complexityfor this algorithm, without additional assumptions on the data-generatingmodel. In particular, we prove that the algorithm incurs an$\tilde{\mathcal{O}}(\delta_{\mathcal{F}} H^2 \sqrt{T})$ regret, where$\delta_{\mathcal{F}}$ characterizes the intrinsic complexity of the functionclass $\mathcal{F}$, $H$ is the length of each episode, and $T$ is the totalnumber of episodes. Our regret bounds are independent of the number of states,a result which exhibits clearly the benefit of function approximation in RL.

On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces