Open AccessAdaptive Neural Network Optimized Control Using Reinforcement Learning of Critic-Actor Architecture for a Class of Non-Affine Nonlinear Systems
Author(s) -
Xue Yang,
Bin Li,
Guoxing Wen
Publication year - 2021
Publication title -
ieee access
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.587
H-Index - 127
ISSN - 2169-3536
DOI - 10.1109/access.2021.3120835
Subject(s) - aerospace , bioengineering , communication, networking and broadcast technologies , components, circuits, devices and systems , computing and processing , engineered materials, dielectrics and plasmas , engineering profession , fields, waves and electromagnetics , general topics for engineers , geoscience , nuclear engineering , photonics and electrooptics , power, energy and industry applications , robotics and control systems , signal processing and analysis , transportation
In this article, an optimized tracking control using critic-actor reinforcement learning (RL) strategy is investigated for a class of non-affine nonlinear continuous-time systems. Since the non-affine system is with the implicit control input in dynamic equation, it is a more general modeling form than the affine case, hence this also makes the optimized control more challenging and rewarding. However, most existing RL-based optimal controllers are very complex in algorithm because their actor and critic training laws obtained by implementing gradient descent on the square of Bellman residual error, which equals to the approximation of Hamilton-Jacobi-Bellman (HJB) equation, hence these methods are difficult to be extended to non-affine systems. In this optimized control, the RL algorithm is produced from implementing gradient descent to a simple positive-definite function, which is derived from HJB equation’s partial derivative. As a result, the proposed control algorithm can be significantly simple so as to alleviate the computational burden. Finally, a typical numerical simulation is carried out, and the results also further confirm effectiveness of the proposed control scheme.