English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper presents new \emph{variance-aware} confidence sets for linearbandits and linear mixture Markov Decision Processes (MDPs). With the newconfidence sets, we obtain the follow regret bounds: For linear bandits, weobtain an $\tilde{O}(poly(d)\sqrt{1 + \sum_{k=1}^{K}\sigma_k^2})$data-dependent regret bound, where $d$ is the feature dimension, $K$ is thenumber of rounds, and $\sigma_k^2$ is the \emph{unknown} variance of the rewardat the $k$-th round. This is the first regret bound that only scales with thevariance and the dimension but \emph{no explicit polynomial dependency on $K$}.When variances are small, this bound can be significantly smaller than the$\tilde{\Theta}\left(d\sqrt{K}\right)$ worst-case regret bound. For linearmixture MDPs, we obtain an $\tilde{O}(poly(d, \log H)\sqrt{K})$ regret bound,where $d$ is the number of base models, $K$ is the number of episodes, and $H$is the planning horizon. This is the first regret bound that only scales\emph{logarithmically} with $H$ in the reinforcement learning with linearfunction approximation setting, thus \emph{exponentially improving} existingresults, and resolving an open problem in \citep{zhou2020nearly}. We developthree technical ideas that may be of independent interest: 1) applications ofthe peeling technique to both the input norm and the variance magnitude, 2) arecursion-based estimator for the variance, and 3) a new convex potential lemmathat generalizes the seminal elliptical potential lemma.

Improved Variance-Aware Confidence Sets for Linear Bandits and Linear Mixture MDP