Premium
Implicit dual control based on particle filtering and forward dynamic programming
Author(s) -
Bayard David S.,
Schumitzky Alan
Publication year - 2010
Publication title -
international journal of adaptive control and signal processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.73
H-Index - 66
eISSN - 1099-1115
pISSN - 0890-6327
DOI - 10.1002/acs.1094
Subject(s) - dynamic programming , dual (grammatical number) , computer science , control theory (sociology) , particle filter , inverted pendulum , stochastic control , block (permutation group theory) , mathematical optimization , sampling (signal processing) , bellman equation , position (finance) , control (management) , optimal control , stochastic programming , function (biology) , filter (signal processing) , mathematics , algorithm , artificial intelligence , nonlinear system , economics , art , literature , biology , geometry , quantum mechanics , evolutionary biology , computer vision , physics , finance
This paper develops a sampling‐based approach to implicit dual control. Implicit dual control methods synthesize stochastic control policies by systematically approximating the stochastic dynamic programming equations of Bellman, in contrast to explicit dual control methods that artificially induce probing into the control law by modifying the cost function to include a term that rewards learning. The proposed implicit dual control approach is novel in that it combines a particle filter with a policy‐iteration method for forward dynamic programming. The integration of the two methods provides a complete sampling‐based approach to the problem. Implementation of the approach is simplified by making use of a specific architecture denoted as a H‐block. Practical suggestions are given for reducing computational loads within the H‐block for real‐time applications. As an example, the method is applied to the control of a stochastic pendulum model having unknown mass, length, initial position and velocity, and unknown sign of its dc gain. Simulation results indicate that active controllers based on the described method can systematically improve closed‐loop performance with respect to other more common stochastic control approaches. Copyright © 2008 John Wiley & Sons, Ltd.