English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Offline or batch reinforcement learning seeks to learn a near-optimal policyusing history data without active exploration of the environment. To counterthe insufficient coverage and sample scarcity of many offline datasets, theprinciple of pessimism has been recently introduced to mitigate high bias ofthe estimated values. While pessimistic variants of model-based algorithms(e.g., value iteration with lower confidence bounds) have been theoreticallyinvestigated, their model-free counterparts -- which do not require explicitmodel estimation -- have not been adequately studied, especially in terms ofsample efficiency. To address this inadequacy, we study a pessimistic variantof Q-learning in the context of finite-horizon Markov decision processes, andcharacterize its sample complexity under the single-policy concentrabilityassumption which does not require the full coverage of the state-action space.In addition, a variance-reduced pessimistic Q-learning algorithm is proposed toachieve near-optimal sample complexity. Altogether, this work highlights theefficiency of model-free algorithms in offline RL when used in conjunction withpessimism and variance reduction.

Pessimistic Q-Learning for Offline Reinforcement Learning: Towards Optimal Sample Complexity