Open AccessAdaptive embedded control of cyber‐physical systems using reinforcement learning
Author(s) -
Mirzaei Buini Hamid,
Peter Steffen,
Givargis Tony
Publication year - 2017
Publication title -
iet cyber‐physical systems: theory and applications
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.308
H-Index - 7
ISSN - 2398-3396
DOI - 10.1049/iet-cps.2017.0048
Subject(s) - reinforcement learning , cyber physical system , computer science , controller (irrigation) , overhead (engineering) , control system , adaptive control , control (management) , control engineering , distributed computing , artificial intelligence , engineering , operating system , electrical engineering , agronomy , biology
Embedded control parameters of cyber‐physical systems (CPS), such as sampling rate, are typically invariant and designed with a worst case scenario in mind. In an over‐engineered system, control parameters are assigned values that satisfy system‐wide performance requirements at the expense of excessive energy and resource overheads. Dynamic and adaptive control parameters can reduce the overhead but are complex and require in‐depth knowledge of the CPS and its operating environment – which typically is unavailable during design time. The authors investigate the application of reinforcement learning (RL) to dynamically adapt high‐level system parameters, at run time, as a function of the system state. RL is an alternative approach to the classical control theory for CPSs that can learn and adapt control properties without the need of an in‐depth controller model. Specifically, we show that RL can modulate sampling times to save processing power without compromising control quality. We apply a novel statistical cloud‐based evaluation framework to study the validity of our approach for the cart‐pole balancing control problem as well as the well‐known mountain car problem. The results show an improved real‐world power efficiency of up to 20% compared with an optimal system with fixed controller settings.