Open AccessUsing reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics
Author(s) -
Zhu Yuanheng,
Zhao Dongbin,
Li Xiangjun
Publication year - 2016
Publication title -
iet control theory and applications
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.059
H-Index - 108
eISSN - 1751-8652
pISSN - 1751-8644
DOI - 10.1049/iet-cta.2015.0769
Subject(s) - reinforcement learning , convergence (economics) , computer science , tracking (education) , control theory (sociology) , adaptation (eye) , dynamic programming , system dynamics , linear programming , lyapunov function , mathematical optimization , nonlinear system , mathematics , artificial intelligence , algorithm , control (management) , psychology , pedagogy , physics , quantum mechanics , optics , economics , economic growth
The optimal tracking of non‐linear systems without knowing system dynamics is an important and intractable problem. Based on the framework of reinforcement learning (RL) and adaptive dynamic programming, a model‐free adaptive optimal tracking algorithm is proposed in this study. After constructing an augmented system with the tracking errors and the reference states, the tracking problem is converted to a regulation problem with respect to the new system. Several RL techniques are synthesised to form a novel algorithm which learns the optimal solution online in real time without any information of the system dynamics. Continuous adaptation laws are defined by the current observations and the past experience. The convergence is guaranteed by Lyapunov analysis. Two simulations on a linear and a non‐linear systems demonstrate the performance of the proposed approach.