Premium
Control of continuous‐time Markov chains with safety constraints
Author(s) -
Hsu ShunPin
Publication year - 2012
Publication title -
international journal of robust and nonlinear control
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.361
H-Index - 106
eISSN - 1099-1239
pISSN - 1049-8923
DOI - 10.1002/rnc.1708
Subject(s) - polyhedron , markov chain , invariant (physics) , probability distribution , intersection (aeronautics) , mathematics , finite set , upper and lower bounds , set (abstract data type) , markov decision process , distribution (mathematics) , state (computer science) , markov process , mathematical optimization , discrete mathematics , computer science , combinatorics , algorithm , mathematical analysis , statistics , engineering , mathematical physics , programming language , aerospace engineering
SUMMARY In this work the controlled continuous‐time finite‐state Markov chain with safety constraints is studied. The constraints are expressed as a finite number of inequalities, whose intersection forms a polyhedron. A probability distribution vector is called safe if it is in the polyhedron. Under the assumptions that the controlled Markov chain is completely observable and the controller induces a unique stationary distribution in the interior of the polyhedron, the author identifies the supreme invariant safety set (SISS) where a set is called an invariant safety set if any probability distribution in the set is initially safe and remains safe as time evolves. In particular, the necessary and sufficient condition for the SISS to be the polyhedron itself is given via linear programming formulations. A closed‐form expression for the condition is also derived as the constraints impose only upper and/or lower bounds on the components of the distribution vectors. If the condition is not satisfied, a finite time bound is identified and used to characterize the SISS. Numerical examples are provided to illustrate the results. Copyright © 2011 John Wiley & Sons, Ltd.