Premium
Forward recursion for markov decision processes with skip‐free‐to‐the‐right transitions part ii: non‐standard applications
Author(s) -
Wijngaard J.,
Stidham S.
Publication year - 2000
Publication title -
statistica neerlandica
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.52
H-Index - 39
eISSN - 1467-9574
pISSN - 0039-0402
DOI - 10.1111/1467-9574.00134
Subject(s) - recursion (computer science) , markov decision process , upper and lower bounds , countable set , mathematics , bellman equation , markov chain , function (biology) , markov process , uniformization (probability theory) , state (computer science) , queue , section (typography) , queueing theory , value (mathematics) , discrete mathematics , mathematical optimization , computer science , markov model , algorithm , markov property , statistics , mathematical analysis , evolutionary biology , programming language , biology , operating system
This paper is the sequel to Wijngaard and Stidham (1986). The topic is a countable state average reward semi‐Markov decision process with a transition mechanism that is skip‐free to the right. The applications are controlled GI /M/1 queues. Skip‐free to the right means that state n cannot be reached from the states i (< n ) without reaching first state n −1. In such decision processes the reversed optimality equation can be used to estimate the optimal average reward by bi‐section. Wijngaard and Stidham (1986) show that it is possible to use upper and lower bounds on the value function for this bi‐section. This paper considers queueing problems where this bi‐section can not be used in a standard way. Instead of an upper bound on the value function is is possible now to use the character of the optimal strategy.