English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Robust Markov Decision Processes (RMDPs) are a widely used framework forsequential decision-making under parameter uncertainty. RMDPs have beenextensively studied when the objective is to maximize the discounted return,but little is known for average optimality (optimizing the long-run average ofthe rewards obtained over time) and Blackwell optimality (remaining discountoptimal for all discount factors sufficiently close to 1). In this paper, weprove several foundational results for RMDPs beyond the discounted return. Weshow that average optimal policies can be chosen stationary and deterministicfor sa-rectangular RMDPs but, perhaps surprisingly, that history-dependent(Markovian) policies strictly outperform stationary policies for averageoptimality in s-rectangular RMDPs. We also study Blackwell optimality forsa-rectangular RMDPs, where we show that {\em approximate} Blackwell optimalpolicies always exist, although Blackwell optimal policies may not exist. Wealso provide a sufficient condition for their existence, which encompassesvirtually any examples from the literature. We then discuss the connectionbetween average and Blackwell optimality, and we describe several algorithms tocompute the optimal average return. Interestingly, our approach leverages theconnections between RMDPs and stochastic games.

Beyond discounted returns: Robust Markov decision processes with average and Blackwell optimality