Open AccessAnalysis of the deployment strategies of reinforcement learning controllers for complex dynamic systems
Author(s) -
Ibrahim Ahmed,
Marcos Q'uiñones Grueiro,
Gautam Biswas
Publication year - 2021
Publication title -
proceedings of the annual conference of the prognostics and health management society
Language(s) - English
Resource type - Journals
ISSN - 2325-0178
DOI - 10.36001/phmconf.2021.v13i1.3020
Subject(s) - reinforcement learning , inefficiency , software deployment , computer science , controller (irrigation) , control (management) , sample (material) , control engineering , artificial intelligence , engineering , chemistry , chromatography , agronomy , economics , biology , microeconomics , operating system
This paper benchmarks several strategies for deploying reinforcement learning (RL)-based controllers on heterogeneous hybrid systems. Sample inefficiency is often a significant cost for RL controllers because we need sufficient data to train them, and the controllers may take time to converge to an acceptable control policy. This can be doubly costly if system health is degrading, or if the network of such systems in turn cannot afford a gradually improving controller in its constituents. Learning speed improvement can be achieved via transfer learning across controllers trained on different tasks: simulations, data-driven models, or separate instances of similar systems. This paper discusses near- and far- transfers across tasks of varying similarities. These approaches are applied on a test-bed of models of cooling towers operating on office and residential buildings on a university campus.