English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Mean Field Control (MFC) is a powerful approximation tool to solvelarge-scale Multi-Agent Reinforcement Learning (MARL) problems. However, thesuccess of MFC relies on the presumption that given the local states andactions of all the agents, the next (local) states of the agents evolveconditionally independent of each other. Here we demonstrate that even in aMARL setting where agents share a common global state in addition to theirlocal states evolving conditionally independently (thus introducing acorrelation between the state transition processes of individual agents), theMFC can still be applied as a good approximation tool. The global state isassumed to be non-decomposable i.e., it cannot be expressed as a collection oflocal states of the agents. We compute the approximation error as$\mathcal{O}(e)$ where $e=\frac{1}{\sqrt{N}}\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]$. The size of the agent population is denoted bythe term $N$, and $|\mathcal{X}|, |\mathcal{U}|$ respectively indicate thesizes of (local) state and action spaces of individual agents. Theapproximation error is found to be independent of the size of the shared globalstate space. We further demonstrate that in a special case if the reward andstate transition functions are independent of the action distribution of thepopulation, then the error can be improved to$e=\frac{\sqrt{|\mathcal{X}|}}{\sqrt{N}}$. Finally, we devise a Natural PolicyGradient based algorithm that solves the MFC problem with$\mathcal{O}(\epsilon^{-3})$ sample complexity and obtains a policy that iswithin $\mathcal{O}(\max\{e,\epsilon\})$ error of the optimal MARL policy forany $\epsilon>0$.

Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global State