English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper presents two algorithms, AdaOFUL and VARA, for online sequentialdecision-making in the presence of heavy-tailed rewards with only finitevariances. For linear stochastic bandits, we address the issue of heavy-tailedrewards by modifying the adaptive Huber regression and proposing AdaOFUL.AdaOFUL achieves a state-of-the-art regret bound of$\widetilde{O}\big(d\big(\sum_{t=1}^T \nu_{t}^2\big)^{1/2}+d\big)$ as if therewards were uniformly bounded, where $\nu_{t}^2$ is the observed conditionalvariance of the reward at round $t$, $d$ is the feature dimension, and$\widetilde{O}(\cdot)$ hides logarithmic dependence. Building upon AdaOFUL, wepropose VARA for linear MDPs, which achieves a tighter variance-aware regretbound of $\widetilde{O}(d\sqrt{HG^*K})$. Here, $H$ is the length of episodes,$K$ is the number of episodes, and $G^*$ is a smaller instance-dependentquantity that can be bounded by other instance-dependent quantities whenadditional structural conditions on the MDP are satisfied. Our regret bound issuperior to the current state-of-the-art bounds in three ways: (1) it dependson a tighter instance-dependent quantity and has optimal dependence on $d$ and$H$, (2) we can obtain further instance-dependent bounds of $G^*$ underadditional structural conditions on the MDP, and (3) our regret bound is valideven when rewards have only finite variances, achieving a level of generalityunmatched by previous works. Overall, our modified adaptive Huber regressionalgorithm may serve as a useful building block in the design of algorithms foronline problems with heavy-tailed rewards.

Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards