English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The robust constrained Markov decision process (RCMDP) is a recenttask-modelling framework for reinforcement learning that incorporatesbehavioural constraints and that provides robustness to errors in thetransition dynamics model through the use of an uncertainty set. SimulatingRCMDPs requires computing the worst-case dynamics based on value estimates foreach state, an approach which has previously been used in the RobustConstrained Policy Gradient (RCPG). Highlighting potential downsides of RCPGsuch as not robustifying the full constrained objective and the lack ofincremental learning, this paper introduces two algorithms, called RCPG withRobust Lagrangian and Adversarial RCPG. RCPG with Robust Lagrangian modifiesRCPG by taking the worst-case dynamics based on the Lagrangian rather thaneither the value or the constraint. Adversarial RCPG also formulates theworst-case dynamics based on the Lagrangian but learns this directly andincrementally as an adversarial policy through gradient descent rather thanindirectly and abruptly through constrained optimisation on a sorted valuelist. A theoretical analysis first derives the Lagrangian policy gradient forthe policy optimisation of both proposed algorithms and then the adversarialpolicy gradient to learn the adversary for Adversarial RCPG. Empiricalexperiments injecting perturbations in inventory management and safe navigationtasks demonstrate the competitive performance of both algorithms compared totraditional RCPG variants as well as non-robust and non-constrained ablations.In particular, Adversarial RCPG ranks among the top two performing algorithmson all tests.

Robust Lagrangian and Adversarial Policy Gradient for Robust Constrained Markov Decision Processes