English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent studies have shown that episodic reinforcement learning (RL) is noharder than bandits when the total reward is bounded by $1$, and proved regretbounds that have a polylogarithmic dependence on the planning horizon $H$.However, it remains an open question that if such results can be carried overto adversarial RL, where the reward is adversarially chosen at each episode. Inthis paper, we answer this question affirmatively by proposing the firsthorizon-free policy search algorithm. To tackle the challenges caused byexploration and adversarially chosen reward, our algorithm employs (1) avariance-uncertainty-aware weighted least square estimator for the transitionkernel; and (2) an occupancy measure-based technique for the online search of a\emph{stochastic} policy. We show that our algorithm achieves an$\tilde{O}\big((d+\log (|\mathcal{S}|^2 |\mathcal{A}|))\sqrt{K}\big)$ regretwith full-information feedback, where $d$ is the dimension of a known featuremapping linearly parametrizing the unknown transition kernel of the MDP, $K$ isthe number of episodes, $|\mathcal{S}|$ and $|\mathcal{A}|$ are thecardinalities of the state and action spaces. We also provide hardness resultsand regret lower bounds to justify the near optimality of our algorithm and theunavoidability of $\log|\mathcal{S}|$ and $\log|\mathcal{A}|$ in the regretbound.

Horizon-free Reinforcement Learning in Adversarial Linear Mixture MDPs