Ergodic Rate Control Problem for Single Class Queueing Networks

We consider critically loaded single class queueing networks with infinite buffers in which arrival and service rates are state (i.e., queue length) dependent and may be dynamically controlled. An optimal rate control problem for such networks with an ergodic cost criterion is studied. It is shown that the value function (i.e., optimum value of the cost) of the rate control problem for the network converges, under a suitable heavy traffic scaling limit, to that of an ergodic control problem for certain controlled reflected diffusions. Furthermore, we show that near optimal controls for limit diffusion models can be used to construct asymptotically near optimal rate control policies for the underlying physical queueing networks. The expected cost per unit time criterion studied here is given in terms of an unbounded holding cost and a linear control cost (“cost for effort”). Time asymptotics of a related uncontrolled model are studied as well. We establish convergence of invariant measures of scaled queue ...