English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we consider a multi-armed bandit in which each arm is a Markovprocess evolving on a finite state space. The state space is common across thearms, and the arms are independent of each other. The transition probabilitymatrix of one of the arms (the odd arm) is different from the common transitionprobability matrix of all the other arms. A decision maker, who knows thesetransition probability matrices, wishes to identify the odd arm as quickly aspossible, while keeping the probability of decision error small. To do so, thedecision maker collects observations from the arms by pulling the arms in asequential manner, one at each discrete time instant. However, the decisionmaker has a trembling hand, and the arm that is actually pulled at any giventime differs, with a small probability, from the one he intended to pull. Theobservation at any given time is the arm that is actually pulled and itscurrent state. The Markov processes of the unobserved arms continue to evolve.This makes the arms restless.  For the above setting, we derive the first known asymptotic lower bound onthe expected time required to identify the odd arm, where the asymptotics is ofvanishing error probability. The continued evolution of each arm adds a newdimension to the problem, leading to a family of Markov decision problems(MDPs) on a countable state space. We then stitch together certainparameterised solutions to these MDPs and obtain a sequence of strategies whoseexpected times to identify the odd arm come arbitrarily close to the lowerbound in the regime of vanishing error probability. Prior works dealt withindependent and identically distributed (across time) arms and rested Markovarms, whereas our work deals with restless Markov arms.

Detecting an Odd Restless Markov Arm with a Trembling Hand