Open AccessResilient adaptive optimal control of distributed multi‐agent systems using reinforcement learning
Author(s) -
Moghadam Rohollah,
Modares Hamidreza
Publication year - 2018
Publication title -
iet control theory and applications
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.059
H-Index - 108
eISSN - 1751-8652
pISSN - 1751-8644
DOI - 10.1049/iet-cta.2018.0029
Subject(s) - reinforcement learning , computer science , controller (irrigation) , protocol (science) , multi agent system , observer (physics) , adversarial system , control (management) , optimal control , distributed computing , state (computer science) , consensus , algebraic riccati equation , control theory (sociology) , artificial intelligence , riccati equation , mathematical optimization , mathematics , algorithm , medicine , mathematical analysis , physics , alternative medicine , pathology , quantum mechanics , agronomy , biology , differential equation
This study presents a unified resilient model‐free reinforcement learning (RL) based distributed control protocol for leader‐follower multi‐agent systems. Although RL has been successfully used to learn optimal control protocols for multi‐agent systems, the effects of adversarial inputs are ignored. It is shown in this study, however, that their adverse effects can propagate across the network and impact the learning outcome of other intact agents. To alleviate this problem, a unified RL‐based distributed control frameworks is developed for both homogeneous and heterogeneous multi‐agent systems to prevent corrupted sensory data from propagating across the network. To this end, only the leader communicates its actual sensory information and other agents estimate the leader' state using a distributed observer and communicate this estimation to their neighbours to achieve consensus on the leader state. The observer cannot be physically affected by any adversarial input. To further improve resiliency, distributedH ∞ control protocols are designed to attenuate the effect of the adversarial inputs on the compromised agent itself. An off‐policy RL algorithm is developed to learn the solutions of the game algebraic Riccati equations arising from solving theH ∞control problem. No knowledge of the agent's dynamics is required and it is shown that the proposed RL‐basedH ∞control protocol is resilient against adversarial inputs.