English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider the problem of finding stationary Nash equilibria (NE) in afinite discounted general-sum stochastic game. We first generalize a non-linearoptimization problem from Filar and Vrieze [2004] to a $N$-player setting andbreak down this problem into simpler sub-problems that ensure there is noBellman error for a given state and an agent. We then provide acharacterization of solution points of these sub-problems that correspond toNash equilibria of the underlying game and for this purpose, we derive a set ofnecessary and sufficient SG-SP (Stochastic Game - Sub-Problem) conditions.Using these conditions, we develop two actor-critic algorithms: OFF-SGSP(model-based) and ON-SGSP (model-free). Both algorithms use a critic thatestimates the value function for a fixed policy and an actor that performsdescent in the policy space using a descent direction that avoids local minima.We establish that both algorithms converge, in self-play, to the equilibria ofa certain ordinary differential equation (ODE), whose stable limit pointscoincide with stationary NE of the underlying general-sum stochastic game. On asingle state non-generic game (see Hart and Mas-Colell [2005]) as well as on asynthetic two-player game setup with $810,000$ states, we establish thatON-SGSP consistently outperforms NashQ ([Hu and Wellman, 2003] and FFQ[Littman, 2001] algorithms.

Actor-Critic Algorithms for Learning Nash Equilibria in N-player General-Sum Games