English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Regret analysis is challenging in Multi-Agent Reinforcement Learning (MARL)primarily due to the dynamical environments and the decentralized informationamong agents. We attempt to solve this challenge in the context ofdecentralized learning in multi-agent linear-quadratic (LQ) dynamical systems.We begin with a simple setup consisting of two agents and two dynamicallydecoupled stochastic linear systems, each system controlled by an agent. Thesystems are coupled through a quadratic cost function. When both systems'dynamics are unknown and there is no communication among the agents, we showthat no learning policy can generate sub-linear in $T$ regret, where $T$ is thetime horizon. When only one system's dynamics are unknown and there isone-directional communication from the agent controlling the unknown system tothe other agent, we propose a MARL algorithm based on the construction of anauxiliary single-agent LQ problem. The auxiliary single-agent problem in theproposed MARL algorithm serves as an implicit coordination mechanism among thetwo learning agents. This allows the agents to achieve a regret within$O(\sqrt{T})$ of the regret of the auxiliary single-agent problem.Consequently, using existing results for single-agent LQ regret, our algorithmprovides a $\tilde{O}(\sqrt{T})$ regret bound. (Here $\tilde{O}(\cdot)$ hidesconstants and logarithmic factors). Our numerical experiments indicate thatthis bound is matched in practice. From the two-agent problem, we extend ourresults to multi-agent LQ systems with certain communication patterns.

Regret Bounds for Decentralized Learning in Cooperative Multi-Agent Dynamical Systems