English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Finding unified complexity measures and algorithms for sample-efficientlearning is a central topic of research in reinforcement learning (RL). TheDecision-Estimation Coefficient (DEC) is recently proposed by Foster et al.(2021) as a necessary and sufficient complexity measure for sample-efficientno-regret RL. This paper makes progress towards a unified theory for RL withthe DEC framework. First, we propose two new DEC-type complexity measures:Explorative DEC (EDEC), and Reward-Free DEC (RFDEC). We show that they arenecessary and sufficient for sample-efficient PAC learning and reward-freelearning, thereby extending the original DEC which only captures no-regretlearning. Next, we design new unified sample-efficient algorithms for all threelearning goals. Our algorithms instantiate variants of theEstimation-To-Decisions (E2D) meta-algorithm with a strong and general modelestimation subroutine. Even in the no-regret setting, our algorithm E2D-TAimproves upon the algorithms of Foster et al. (2021) which require eitherbounding a variant of the DEC which may be prohibitively large, or designingproblem-specific estimation subroutines. As applications, we recover existingand obtain new sample-efficient learning results for a wide range of tractableRL problems using essentially a single algorithm. We also generalize the DEC togive sample-efficient algorithms for all-policy model estimation, withapplications for learning equilibria in Markov Games. Finally, as a connection,we re-analyze two existing optimistic model-based algorithms based on PosteriorSampling or Maximum Likelihood Estimation, showing that they enjoy similarregret bounds as E2D-TA under similar structural conditions as the DEC.

Unified Algorithms for RL with Decision-Estimation Coefficients: No-Regret, PAC, and Reward-Free Learning