English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Actor Critic methods have found immense applications on a wide range ofReinforcement Learning tasks especially when the state-action space is large.In this paper, we consider actor critic and natural actor critic algorithmswith function approximation for constrained Markov decision processes (C-MDP)involving inequality constraints and carry out a non-asymptotic analysis forboth of these algorithms in a non-i.i.d (Markovian) setting. We consider thelong-run average cost criterion where both the objective and the constraintfunctions are suitable policy-dependent long-run averages of certain prescribedcost functions. We handle the inequality constraints using the Lagrangemultiplier method. We prove that these algorithms are guaranteed to find afirst-order stationary point (i.e., $\Vert \nabla L(\theta,\gamma)\Vert_2^2\leq \epsilon$) of the performance (Lagrange) function $L(\theta,\gamma)$, witha sample complexity of $\mathcal{\tilde{O}}(\epsilon^{-2.5})$ in the case ofboth Constrained Actor Critic (C-AC) and Constrained Natural Actor Critic(C-NAC) algorithms.We also show the results of experiments on a few differentgrid world settings and observe good empirical performance using both of thesealgorithms. In particular, for large grid sizes, Constrained Natural ActorCritic shows slightly better results than Constrained Actor Critic while thelatter is slightly better for a small grid size.

Finite Time Analysis of Constrained Actor Critic and Constrained Natural Actor Critic Algorithms