A reinforcement learning‐based scheme for direct adaptive optimal control of linear stochastic systems

Reinforcement learning where decision‐making agents learn optimal policies through environmental interactions is an attractive paradigm for model‐free, adaptive controller design. However, results for systems with continuous state and action variables are rare. In this paper, we present convergence results for optimal linear quadratic control of discrete‐time linear stochastic systems. This work can be viewed as a generalization of a previous work on deterministic linear systems. Key differences between the algorithms for deterministic and stochastic systems are highlighted through examples. The usefulness of the algorithm is demonstrated through a nonlinear chemostat bioreactor case study. Copyright © 2009 John Wiley & Sons, Ltd.