English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

We analyze Gittins' Markovian model, as generalized by Varaiya, Wal- rand and Buyukkoc, in discrete and continuous time. The approach resem- bles Weber's modication of Whittle's, within the framework of both multi- parameter processes and excursion theory. It is shown that index-priority strategies are optimal, in concert with all the special cases that have been treated previously. 1. Introduction. A multi-armed bandit is a control model that supports dynamic allocation of scarce resources in the face of uncertainty (2, 3, 8, 15). Each arm of the bandit represents an ongoing project and pulling arms corre- sponds to allocating resources among the projects. In a discrete-time model, arms are pulled one at a time and each pull results in a reward. This is in contrast to continuous time, where a more appropriate view is that of a re- source (time, effort) which is to be allocated simultaneously among the arms while accruing rewards continuously. The goal is to identify optimal allocation strategies, and in this paper it is achieved for bandits with independent arms and random rewards, discounted over an innite horizon. Specically , we analyze Gittins's Markovian model (9, 8) in discrete and con- tinuous time, as generalized by (18) and (13) in the spirit of (17). The approach resembles Weber's (20) modication of (21) (see also (5{7)), within the frame-

Multi-armed bandits in discrete and continuous time