English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study non-parametric estimation of the value function of aninfinite-horizon $\gamma$-discounted Markov reward process (MRP) usingobservations from a single trajectory. We provide non-asymptotic guarantees fora general family of kernel-based multi-step temporal difference (TD) estimates,including canonical $K$-step look-ahead TD for $K = 1, 2, \ldots$ and theTD$(\lambda)$ family for $\lambda \in [0,1)$ as special cases. Our boundscapture its dependence on Bellman fluctuations, mixing time of the Markovchain, any mis-specification in the model, as well as the choice of weightfunction defining the estimator itself, and reveal some delicate interactionsbetween mixing time and model mis-specification. For a given TD method appliedto a well-specified model, its statistical error under trajectory data issimilar to that of i.i.d. sample transition pairs, whereas undermis-specification, temporal dependence in data inflates the statistical error.However, any such deterioration can be mitigated by increased look-ahead. Wecomplement our upper bounds by proving minimax lower bounds that establishoptimality of TD-based methods with appropriately chosen look-ahead andweighting, and reveal some fundamental differences between value functionestimation and ordinary non-parametric regression.

Policy evaluation from a single path: Multi-step methods, mixing and mis-specification