English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper studies safe Reinforcement Learning (safe RL) with linear functionapproximation and under hard instantaneous constraints where unsafe actionsmust be avoided at each step. Existing studies have considered safe RL withhard instantaneous constraints, but their approaches rely on several keyassumptions: $(i)$ the RL agent knows a safe action set for {\it every} stateor knows a {\it safe graph} in which all the state-action-state triples aresafe, and $(ii)$ the constraint/cost functions are {\it linear}. In this paper,we consider safe RL with instantaneous hard constraints without assumption$(i)$ and generalize $(ii)$ to Reproducing Kernel Hilbert Space (RKHS). Ourproposed algorithm, LSVI-AE, achieves $\tilde{\cO}(\sqrt{d^3H^4K})$ regret and$\tilde{\cO}(H \sqrt{dK})$ hard constraint violation when the cost function islinear and $\cO(H\gamma_K \sqrt{K})$ hard constraint violation when the costfunction belongs to RKHS. Here $K$ is the learning horizon, $H$ is the lengthof each episode, and $\gamma_K$ is the information gain w.r.t the kernel usedto approximate cost functions. Our results achieve the optimal dependency onthe learning horizon $K$, matching the lower bound we provide in this paper anddemonstrating the efficiency of LSVI-AE. Notably, the design of our approachencourages aggressive policy exploration, providing a unique perspective onsafe RL with general cost functions and no prior knowledge of safe actions,which may be of independent interest.

Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration