English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the problem of off-policy evaluation (OPE) for episodic PartiallyObservable Markov Decision Processes (POMDPs) with continuous states. Motivatedby the recently proposed proximal causal inference framework, we develop anon-parametric identification result for estimating the policy value via asequence of so-called V-bridge functions with the help of time-dependent proxyvariables. We then develop a fitted-Q-evaluation-type algorithm to estimateV-bridge functions recursively, where a non-parametric instrumental variable(NPIV) problem is solved at each step. By analyzing this challenging sequentialNPIV problem, we establish the finite-sample error bounds for estimating theV-bridge functions and accordingly that for evaluating the policy value, interms of the sample size, length of horizon and so-called (local) measure ofill-posedness at each step. To the best of our knowledge, this is the firstfinite-sample error bound for OPE in POMDPs under non-parametric models.

Off-Policy Evaluation for Episodic Partially Observable Markov Decision Processes under Non-Parametric Models