English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider an extension to the restless multi-armed bandit (RMAB) problemwith unknown arm dynamics, where an unknown exogenous global Markov processgoverns the rewards distribution of each arm. Under each global state, therewards process of each arm evolves according to an unknown Markovian rule,which is non-identical among different arms. At each time, a player chooses anarm out of N arms to play, and receives a random reward from a finite set ofreward states. The arms are restless, that is, their local state evolvesregardless of the player's actions. Motivated by recent studies on related RMABsettings, the regret is defined as the reward loss with respect to a playerthat knows the dynamics of the problem, and plays at each time t the arm thatmaximizes the expected immediate value. The objective is to develop anarm-selection policy that minimizes the regret. To that end, we develop theLearning under Exogenous Markov Process (LEMP) algorithm. We analyze LEMPtheoretically and establish a finite-sample bound on the regret. We show thatLEMP achieves a logarithmic regret order with time. We further analyze LEMPnumerically and present simulation results that support the theoreticalfindings and demonstrate that LEMP significantly outperforms alternativealgorithms.

Restless Multi-Armed Bandits under Exogenous Global Markov Process