English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the regret guarantee for risk-sensitive reinforcement learning(RSRL) via distributional reinforcement learning (DRL) methods. In particular,we consider finite episodic Markov decision processes whose objective is theentropic risk measure (EntRM) of return. By leveraging a key property of theEntRM, the independence property, we establish the risk-sensitivedistributional dynamic programming framework. We then propose two novel DRLalgorithms that implement optimism through two different schemes, including amodel-free one and a model-based one.  We prove that they both attain $\tilde{\mathcal{O}}(\frac{\exp(|\beta|H)-1}{|\beta|}H\sqrt{S^2AK})$ regret upper bound, where $S$, $A$, $K$, and $H$represent the number of states, actions, episodes, and the time horizon,respectively. It matches RSVI2 proposed in \cite{fei2021exponential}, withnovel distributional analysis. To the best of our knowledge, this is the firstregret analysis that bridges DRL and RSRL in terms of sample complexity.  Acknowledging the computational inefficiency associated with the model-freeDRL algorithm, we propose an alternative DRL algorithm with distributionrepresentation. This approach not only maintains the established regret boundsbut also significantly amplifies computational efficiency.  We also prove a tighter minimax lower bound of $\Omega(\frac{\exp(\betaH/6)-1}{\beta H}H\sqrt{SAT})$ for the $\beta>0$ case, which recovers the tightlower bound $\Omega(H\sqrt{SAT})$ in the risk-neutral setting.

Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Bounds