English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We revisit the problem of learning in two-player zero-sum Markov games,focusing on developing an algorithm that is uncoupled, convergent, andrational, with non-asymptotic convergence rates. We start from the case ofstateless matrix game with bandit feedback as a warm-up, showing an$O(t^{-\frac{1}{8}})$ last-iterate convergence rate. To the best of ourknowledge, this is the first result that obtains finite last-iterateconvergence rate given access to only bandit feedback. We extend our result tothe case of irreducible Markov games, providing a last-iterate convergence rateof $O(t^{-\frac{1}{9+\varepsilon}})$ for any $\varepsilon>0$. Finally, we studyMarkov games without any assumptions on the dynamics, and show a pathconvergence rate, which is a new notion of convergence we defined, of$O(t^{-\frac{1}{10}})$. Our algorithm removes the coordination and priorknowledge requirement of [Wei et al., 2021], which pursued the same goals as usfor irreducible Markov games. Our algorithm is related to [Chen et al., 2021,Cen et al., 2021] and also builds on the entropy regularization technique.However, we remove their requirement of communications on the entropy values,making our algorithm entirely uncoupled.

Uncoupled and Convergent Learning in Two-Player Zero-Sum Markov Games with Bandit Feedback