English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Multi-agent reinforcement learning (MARL) has attracted much researchattention recently. However, unlike its single-agent counterpart, manytheoretical and algorithmic aspects of MARL have not been well-understood. Inthis paper, we study the emergence of coordinated behavior by autonomous agentsusing an actor-critic (AC) algorithm. Specifically, we propose and analyze aclass of coordinated actor-critic algorithms (CAC) in which individuallyparametrized policies have a {\it shared} part (which is jointly optimizedamong all agents) and a {\it personalized} part (which is only locallyoptimized). Such kind of {\it partially personalized} policy allows agents tolearn to coordinate by leveraging peers' past experience and adapt toindividual tasks. The flexibility in our design allows the proposed MARL-CACalgorithm to be used in a {\it fully decentralized} setting, where the agentscan only communicate with their neighbors, as well as a {\it federated}setting, where the agents occasionally communicate with a server whileoptimizing their (partially personalized) local models. Theoretically, we showthat under some standard regularity assumptions, the proposed MARL-CACalgorithm requires $\mathcal{O}(\epsilon^{-\frac{5}{2}})$ samples to achieve an$\epsilon$-stationary solution (defined as the solution whose squared norm ofthe gradient of the objective function is less than $\epsilon$). To the best ofour knowledge, this work provides the first finite-sample guarantee fordecentralized AC algorithm with partially personalized policies.

Learning to Coordinate in Multi-Agent Systems: A Coordinated Actor-Critic Algorithm and Finite-Time Guarantees