English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Mean field control (MFC) is an effective way to mitigate the curse ofdimensionality of cooperative multi-agent reinforcement learning (MARL)problems. This work considers a collection of $N_{\mathrm{pop}}$ heterogeneousagents that can be segregated into $K$ classes such that the $k$-th classcontains $N_k$ homogeneous agents. We aim to prove approximation guarantees ofthe MARL problem for this heterogeneous system by its corresponding MFCproblem. We consider three scenarios where the reward and transition dynamicsof all agents are respectively taken to be functions of $(1)$ joint state andaction distributions across all classes, $(2)$ individual distributions of eachclass, and $(3)$ marginal distributions of the entire population. We show that,in these cases, the $K$-class MARL problem can be approximated by MFC witherrors given as$e_1=\mathcal{O}(\frac{\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}}{N_{\mathrm{pop}}}\sum_{k}\sqrt{N_k})$,$e_2=\mathcal{O}(\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]\sum_{k}\frac{1}{\sqrt{N_k}})$and$e_3=\mathcal{O}\left(\left[\sqrt{|\mathcal{X}|}+\sqrt{|\mathcal{U}|}\right]\left[\frac{A}{N_{\mathrm{pop}}}\sum_{k\in[K]}\sqrt{N_k}+\frac{B}{\sqrt{N_{\mathrm{pop}}}}\right]\right)$,respectively, where $A, B$ are some constants and $|\mathcal{X}|,|\mathcal{U}|$are the sizes of state and action spaces of each agent. Finally, we design aNatural Policy Gradient (NPG) based algorithm that, in the three cases statedabove, can converge to an optimal MARL policy within $\mathcal{O}(e_j)$ errorwith a sample complexity of $\mathcal{O}(e_j^{-3})$, $j\in\{1,2,3\}$,respectively.

On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)