English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reinforcement learning is an attractive approach to learn good resourceallocation and scheduling policies based on data when the system model isunknown. However, the cumulative regret of most RL algorithms scales as $\tildeO(\mathsf{S} \sqrt{\mathsf{A} T})$, where $\mathsf{S}$ is the size of the statespace, $\mathsf{A}$ is the size of the action space, $T$ is the horizon, andthe $\tilde{O}(\cdot)$ notation hides logarithmic terms. Due to the lineardependence on the size of the state space, these regret bounds areprohibitively large for resource allocation and scheduling problems. In thispaper, we present a model-based RL algorithm for such problems which hasscalable regret. In particular, we consider a restless bandit model, andpropose a Thompson-sampling based learning algorithm which is tuned to theunderlying structure of the model. We present two characterizations of theregret of the proposed algorithm with respect to the Whittle index policy.First, we show that for a restless bandit with $n$ arms and at most $m$activations at each time, the regret scales either as $\tilde{O}(mn\sqrt{T})$or $\tilde{O}(n^2 \sqrt{T})$ depending on the reward model. Second, under anadditional technical assumption, we show that the regret scales as$\tilde{O}(n^{1.5} \sqrt{T})$ or $\tilde{O}(\max\{m\sqrt{n}, n\} \sqrt{T})$. Wepresent numerical examples to illustrate the salient features of the algorithm.

On learning Whittle index policy for restless bandits with scalable regret