English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study episodic two-player zero-sum Markov games (MGs) in the offlinesetting, where the goal is to find an approximate Nash equilibrium (NE) policypair based on a dataset collected a priori. When the dataset does not haveuniform coverage over all policy pairs, finding an approximate NE involveschallenges in three aspects: (i) distributional shift between the behaviorpolicy and the optimal policy, (ii) function approximation to handle largestate space, and (iii) minimax optimization for equilibrium solving. We proposea pessimism-based algorithm, dubbed as pessimistic minimax value iteration(PMVI), which overcomes the distributional shift by constructing pessimisticestimates of the value functions for both players and outputs a policy pair bysolving NEs based on the two value functions. Furthermore, we establish adata-dependent upper bound on the suboptimality which recovers a sublinear ratewithout the assumption on uniform coverage of the dataset. We also prove aninformation-theoretical lower bound, which suggests that the data-dependentterm in the upper bound is intrinsic. Our theoretical results also highlight anotion of "relative uncertainty", which characterizes the necessary andsufficient condition for achieving sample efficiency in offline MGs. To thebest of our knowledge, we provide the first nearly minimax optimal result foroffline MGs with function approximation.

Pessimistic Minimax Value Iteration: Provably Efficient Equilibrium Learning from Offline Datasets