Premium
Online identifier–actor–critic algorithm for optimal control of nonlinear systems
Author(s) -
Lin Hanquan,
Wei Qinglai,
Liu Derong
Publication year - 2016
Publication title -
optimal control applications and methods
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.458
H-Index - 44
eISSN - 1099-1514
pISSN - 0143-2087
DOI - 10.1002/oca.2259
Subject(s) - identifier , artificial neural network , computer science , bellman equation , optimal control , bounded function , nonlinear system , lyapunov function , dynamic programming , mathematical optimization , affine transformation , stability (learning theory) , control theory (sociology) , control (management) , algorithm , mathematics , artificial intelligence , machine learning , mathematical analysis , physics , quantum mechanics , pure mathematics , programming language
Summary In this paper, a novel identifier–actor–critic optimal control scheme is developed for discrete‐time affine nonlinear systems with uncertainties. In contrast to traditional adaptive dynamic programming methodology, which requires at least partial knowledge of the system dynamics, a neural‐network identifier is employed to learn the unknown control coefficient matrix working together with actor–critic‐based scheme to solve the optimal control online. The critic network learns the approximate value function at each step. The actor network attempts to improve the current policy based on the approximate value function. The weights of all neural networks are updated at each sampling instant. Lyapunov theory is utilized to prove the stability of closed‐loop system. It shows that system states and neural network weights are uniformly ultimately bounded. Finally, simulations are provided to illustrate the effectiveness of the developed method. Copyright © 2016 John Wiley & Sons, Ltd.