English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

It is believed that a model-based approach for reinforcement learning (RL) isthe key to reduce sample complexity. However, the understanding of the sampleoptimality of model-based RL is still largely missing, even for the linearcase. This work considers sample complexity of finding an $\epsilon$-optimalpolicy in a Markov decision process (MDP) that admits a linear additive featurerepresentation, given only access to a generative model. We solve this problemvia a plug-in solver approach, which builds an empirical model and plans inthis empirical model via an arbitrary plug-in solver. We prove that under theanchor-state assumption, which implies implicit non-negativity in the featurespace, the minimax sample complexity of finding an $\epsilon$-optimal policy ina $\gamma$-discounted MDP is $O(K/(1-\gamma)^3\epsilon^2)$, which only dependson the dimensionality $K$ of the feature space and has no dependence on thestate or action space. We further extend our results to a relaxed setting whereanchor-states may not exist and show that a plug-in approach can be sampleefficient as well, providing a flexible approach to design model-basedalgorithms for RL.

Is Plug-in Solver Sample-Efficient for Feature-based Reinforcement Learning?