Premium
MARKOV DECISION PROCESSES WITH UNCERTAIN TRANSITION RATES: SENSITIVITY AND MAX HYPHEN MIN CONTROL
Author(s) -
Kalyanasundaram Suresh,
Chong Edwin K. P.,
Shroff Ness B.
Publication year - 2004
Publication title -
asian journal of control
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.769
H-Index - 53
eISSN - 1934-6093
pISSN - 1561-8625
DOI - 10.1111/j.1934-6093.2004.tb00203.x
Subject(s) - markov decision process , sensitivity (control systems) , markov chain , transition rate matrix , range (aeronautics) , transition (genetics) , markov model , markov process , mathematical optimization , mathematics , decision problem , unit (ring theory) , computer science , statistics , algorithm , engineering , chemistry , biochemistry , electronic engineering , gene , aerospace engineering , mathematics education
Solution techniques for Markov decision problems rely on exact knowledge of the transition rates, which may be difficult or impossible to obtain. In this paper, we consider Markov decision problems with uncertain transition rates represented as compact sets. We first consider the problem of sensitivity analysis where the aim is to quantify the range of uncertainty of the average per‐unit‐time reward given the range of uncertainty of the transition rates. We then develop solution techniques for the problem of obtaining the max‐min optimal policy, which maximizes the worst‐case average per‐unit‐time reward. In each of these problems, we distinguish between systems that can have their transition rates chosen independently and those where the transition rates depend on each other. Our solution techniques are applicable to Markov decision processes with fixed but unknown transition rates and to those with time‐varying transition rates.