English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce SCAL, an algorithm designed to perform efficientexploration-exploitation in any unknown weakly-communicating Markov decisionprocess (MDP) for which an upper bound $c$ on the span of the optimal biasfunction is known. For an MDP with $S$ states, $A$ actions and $\Gamma \leq S$possible next states, we prove a regret bound of $\widetilde{O}(c\sqrt{\GammaSAT})$, which significantly improves over existing algorithms (e.g., UCRL andPSRL), whose regret scales linearly with the MDP diameter $D$. In fact, theoptimal bias span is finite and often much smaller than $D$ (e.g., $D=\infty$in non-communicating MDPs). A similar result was originally derived by Bartlettand Tewari (2009) for REGAL.C, for which no tractable algorithm is available.In this paper, we relax the optimization problem at the core of REGAL.C, wecarefully analyze its properties, and we provide the first computationallyefficient algorithm to solve it. Finally, we report numerical simulationssupporting our theoretical findings and showing how SCAL significantlyoutperforms UCRL in MDPs with large diameter and small span.

Efficient Bias-Span-Constrained Exploration-Exploitation in Reinforcement Learning