English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

This work studies the statistical limits of uniform convergence for offlinepolicy evaluation (OPE) problems with model-based methods (for episodic MDP)and provides a unified framework towards optimal learning for severalwell-motivated offline tasks. Uniform OPE$\sup_\Pi|Q^\pi-\hat{Q}^\pi|<\epsilon$ is a stronger measure than thepoint-wise OPE and ensures offline learning when $\Pi$ contains all policies(the global class). In this paper, we establish an $\Omega(H^2S/d_m\epsilon^2)$ lower bound (over model-based family) for the global uniformOPE and our main result establishes an upper bound of$\tilde{O}(H^2/d_m\epsilon^2)$ for the \emph{local} uniform convergence thatapplies to all \emph{near-empirically optimal} policies for the MDPs with\emph{stationary} transition. Here $d_m$ is the minimal marginal state-actionprobability. Critically, the highlight in achieving the optimal rate$\tilde{O}(H^2/d_m\epsilon^2)$ is our design of \emph{singleton absorbing MDP},which is a new sharp analysis tool that works with the model-based approach. Wegeneralize such a model-based framework to the new settings: offlinetask-agnostic and the offline reward-free with optimal complexity$\tilde{O}(H^2\log(K)/d_m\epsilon^2)$ ($K$ is the number of tasks) and$\tilde{O}(H^2S/d_m\epsilon^2)$ respectively. These results provide a unifiedsolution for simultaneously solving different offline RL problems.

Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settings