English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We propose policy gradient algorithms for robust infinite-horizon Markovdecision processes (MDPs) with non-rectangular uncertainty sets, therebyaddressing an open challenge in the robust MDP literature. Indeed, uncertaintysets that display statistical optimality properties and make optimal use oflimited data often fail to be rectangular. Unfortunately, the correspondingrobust MDPs cannot be solved with dynamic programming techniques and are infact provably intractable. We first present a randomized projected Langevindynamics algorithm that solves the robust policy evaluation problem to globaloptimality but is inefficient. We also propose a deterministic policy gradientmethod that is efficient but solves the robust policy evaluation problem onlyapproximately, and we prove that the approximation error scales with a newmeasure of non-rectangularity of the uncertainty set. Finally, we describe anactor-critic algorithm that finds an $\epsilon$-optimal solution for the robustpolicy improvement problem in $\mathcal{O}(1/\epsilon^4)$ iterations. We thuspresent the first complete solution scheme for robust MDPs with non-rectangularuncertainty sets offering global optimality guarantees. Numerical experimentsshow that our algorithms compare favorably against state-of-the-art methods.

Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Sets