English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Robust Markov Decision Processes (MDPs) and risk-sensitive MDPs are bothpowerful tools for making decisions in the presence of uncertainties. Previousefforts have aimed to establish their connections, revealing equivalences inspecific formulations. This paper introduces a new formulation forrisk-sensitive MDPs, which assesses risk in a slightly different mannercompared to the classical Markov risk measure (Ruszczy\'nski 2010), andestablishes its equivalence with a class of regularized robust MDP (RMDP)problems, including the standard RMDP as a special case. Leveraging thisequivalence, we further derive the policy gradient theorem for both problems,proving gradient domination and global convergence of the exact policy gradientmethod under the tabular setting with direct parameterization. This forms asharp contrast to the Markov risk measure, known to be potentiallynon-gradient-dominant (Huang et al. 2021). We also propose a sample-basedoffline learning algorithm, namely the robust fitted-Z iteration (RFZI), for aspecific regularized RMDP problem with a KL-divergence regularization term (orequivalently the risk-sensitive MDP with an entropy risk measure). We showcaseits streamlined design and less stringent assumptions due to the equivalenceand analyze its sample complexity.

Regularized Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity