English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We introduce the Blackwell discount factor for Markov Decision Processes(MDPs). Classical objectives for MDPs include discounted, average, andBlackwell optimality. Many existing approaches to computing average-optimalpolicies solve for discounted optimal policies with a discount factor close to$1$, but they only work under strong or hard-to-verify assumptions such asergodicity or weakly communicating MDPs. In this paper, we show that when thediscount factor is larger than the Blackwell discount factor$\gamma_{\mathrm{bw}}$, all discounted optimal policies become Blackwell- andaverage-optimal, and we derive a general upper bound on $\gamma_{\mathrm{bw}}$.The upper bound on $\gamma_{\mathrm{bw}}$ provides the first reduction fromaverage and Blackwell optimality to discounted optimality, without anyassumptions, and new polynomial-time algorithms for average- andBlackwell-optimal policies. Our work brings new ideas from the study ofpolynomials and algebraic numbers to the analysis of MDPs. Our results alsoapply to robust MDPs, enabling the first algorithms to compute robustBlackwell-optimal policies.

Reducing Blackwell and Average Optimality to Discounted MDPs via the Blackwell Discount Factor