English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we consider the problem of online learning of Markov decisionprocesses (MDPs) with very large state spaces. Under the assumptions ofrealizable function approximation and low Bellman ranks, we develop an onlinelearning algorithm that learns the optimal value function while at the sametime achieving very low cumulative regret during the learning process. Ourlearning algorithm, Adaptive Value-function Elimination (AVE), is inspired bythe policy elimination algorithm proposed in (Jiang et al., 2017), known asOLIVE. One of our key technical contributions in AVE is to formulate theelimination steps in OLIVE as contextual bandit problems. This techniqueenables us to apply the active elimination and expert weighting methods from(Dudik et al., 2011), instead of the random action exploration scheme used inthe original OLIVE algorithm, for more efficient exploration and better controlof the regret incurred in each policy elimination step. To the best of ourknowledge, this is the first $\sqrt{n}$-regret result for reinforcementlearning in stochastic MDPs with general value function approximation.

$\sqrt{n}$-Regret for Learning in Markov Decision Processes with Function Approximation and Low Bellman Rank