English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study offline reinforcement learning (RL) in partially observable Markovdecision processes. In particular, we aim to learn an optimal policy from adataset collected by a behavior policy which possibly depends on the latentstate. Such a dataset is confounded in the sense that the latent statesimultaneously affects the action and the observation, which is prohibitive forexisting offline RL algorithms. To this end, we propose the \underline{P}roxyvariable \underline{P}essimistic \underline{P}olicy \underline{O}ptimization(\texttt{P3O}) algorithm, which addresses the confounding bias and thedistributional shift between the optimal and behavior policies in the contextof general function approximation. At the core of \texttt{P3O} is a coupledsequence of pessimistic confidence regions constructed via proximal causalinference, which is formulated as minimax estimation. Under a partial coverageassumption on the confounded dataset, we prove that \texttt{P3O} achieves a$n^{-1/2}$-suboptimality, where $n$ is the number of trajectories in thedataset. To our best knowledge, \texttt{P3O} is the first provably efficientoffline RL algorithm for POMDPs with a confounded dataset.

Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes