English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We examine global non-asymptotic convergence properties of policy gradientmethods for multi-agent reinforcement learning (RL) problems in Markovpotential games (MPG). To learn a Nash equilibrium of an MPG in which the sizeof state space and/or the number of players can be very large, we propose newindependent policy gradient algorithms that are run by all players in tandem.When there is no uncertainty in the gradient evaluation, we show that ouralgorithm finds an $\epsilon$-Nash equilibrium with $O(1/\epsilon^2)$ iterationcomplexity which does not explicitly depend on the state space size. When theexact gradient is not available, we establish $O(1/\epsilon^5)$ samplecomplexity bound in a potentially infinitely large state space for asample-based algorithm that utilizes function approximation. Moreover, weidentify a class of independent policy gradient algorithms that enjoysconvergence for both zero-sum Markov games and Markov cooperative games withthe players that are oblivious to the types of games being played. Finally, weprovide computational experiments to corroborate the merits and theeffectiveness of our theoretical developments.

Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence