English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the off-policy evaluation (OPE) problem in an infinite-horizonMarkov decision process with continuous states and actions. We recast the$Q$-function estimation into a special form of the nonparametric instrumentalvariables (NPIV) estimation problem. We first show that under one mildcondition the NPIV formulation of $Q$-function estimation is well-posed in thesense of $L^2$-measure of ill-posedness with respect to the data generatingdistribution, bypassing a strong assumption on the discount factor $\gamma$imposed in the recent literature for obtaining the $L^2$ convergence rates ofvarious $Q$-function estimators. Thanks to this new well-posed property, wederive the first minimax lower bounds for the convergence rates ofnonparametric estimation of $Q$-function and its derivatives in both sup-normand $L^2$-norm, which are shown to be the same as those for the classicalnonparametric regression (Stone, 1982). We then propose a sieve two-stage leastsquares estimator and establish its rate-optimality in both norms under somemild conditions. Our general results on the well-posedness and the minimaxlower bounds are of independent interest to study not only other nonparametricestimators for $Q$-function but also efficient estimation on the value of anytarget policy in off-policy settings.

On Well-posedness and Minimax Optimal Rates of Nonparametric Q-function Estimation in Off-policy Evaluation