Premium
Dynamic intermittent Q ‐learning–based model‐free suboptimal co‐design of L 2 ‐stabilization
Author(s) -
Yang Yongliang,
Vamvoudakis Kyriakos G.,
Ferraz Henrique,
Modares Hamidreza
Publication year - 2019
Publication title -
international journal of robust and nonlinear control
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.361
H-Index - 106
eISSN - 1099-1239
pISSN - 1049-8923
DOI - 10.1002/rnc.4515
Subject(s) - control theory (sociology) , computer science , flexibility (engineering) , stability theory , system dynamics , controller (irrigation) , bandwidth (computing) , transmission (telecommunications) , q learning , feedback controller , nonlinear system , control (management) , mathematics , reinforcement learning , artificial intelligence , computer network , telecommunications , statistics , physics , quantum mechanics , agronomy , biology
Summary This paper proposes an intermittent model‐free learning algorithm for linear time‐invariant systems, where the control policy and transmission decisions are co‐designed simultaneously while also being subjected to worst‐case disturbances. The control policy is designed by introducing an internal dynamical system to further reduce the transmission rate and provide bandwidth flexibility in cyber‐physical systems. Moreover, a Q ‐learning algorithm with two actors and a single critic structure is developed to learn the optimal parameters of a Q ‐function. It is shown by using an impulsive system approach that the closed‐loop system has an asymptotically stable equilibrium and that no Zeno behavior occurs. Furthermore, a qualitative performance analysis of the model‐free dynamic intermittent framework is given and shows the degree of suboptimality concerning the optimal continuous updated controller. Finally, a numerical simulation of an unknown system is carried out to highlight the efficacy of the proposed framework.