Premium
Using process data to generate an optimal control policy via apprenticeship and reinforcement learning
Author(s) -
Mowbray Max,
Smith Robin,
Del RioChaa Ehecatl A.,
Zhang Dongda
Publication year - 2021
Publication title -
aiche journal
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.958
H-Index - 167
eISSN - 1547-5905
pISSN - 0001-1541
DOI - 10.1002/aic.17306
Subject(s) - reinforcement learning , process (computing) , computer science , abstraction , control (management) , function (biology) , process control , identification (biology) , optimal control , artificial intelligence , control function , machine learning , mathematical optimization , mathematics , philosophy , botany , epistemology , evolutionary biology , biology , operating system
Reinforcement learning (RL) is a data‐driven approach to synthesizing an optimal control policy. A barrier to wide implementation of RL‐based controllers is its data‐hungry nature during online training and its inability to extract useful information from human operator and historical process operation data. Here, we present a two‐step framework to resolve this challenge. First, we employ apprenticeship learning via inverse RL to analyze historical process data for synchronous identification of a reward function and parameterization of the control policy. This is conducted offline. Second, the parameterization is improved online efficiently under the ongoing process via RL within only a few iterations. Significant advantages of this framework include to allow for the hot‐start of RL algorithms for process optimal control, and robust abstraction of existing controllers and control knowledge from data. The framework is demonstrated on three case studies, showing its potential for chemical process control.