English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In the trial-and-error mechanism of reinforcement learning (RL), a notoriouscontradiction arises when we expect to learn a safe policy: how to learn a safepolicy without enough data and prior model about the dangerous region? Existingmethods mostly use the posterior penalty for dangerous actions, which meansthat the agent is not penalized until experiencing danger. This fact causesthat the agent cannot learn a zero-violation policy even after convergence.Otherwise, it would not receive any penalty and lose the knowledge aboutdanger. In this paper, we propose the safe set actor-critic (SSAC) algorithm,which confines the policy update using safety-oriented energy functions, or thesafety indexes. The safety index is designed to increase rapidly forpotentially dangerous actions, which allows us to locate the safe set on theaction space, or the control safe set. Therefore, we can identify the dangerousactions prior to taking them, and further obtain a zero constraint-violationpolicy after convergence.We claim that we can learn the energy function in amodel-free manner similar to learning a value function. By using the energyfunction transition as the constraint objective, we formulate a constrained RLproblem. We prove that our Lagrangian-based solutions make sure that thelearned policy will converge to the constrained optimum under some assumptions.The proposed algorithm is evaluated on both the complex simulation environmentsand a hardware-in-loop (HIL) experiment with a real controller from theautonomous vehicle. Experimental results suggest that the converged policy inall environments achieves zero constraint violation and comparable performancewith model-based baselines.

Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learning