English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While deep reinforcement learning has achieved tremendous successes invarious applications, most existing works only focus on maximizing the expectedvalue of total return and thus ignore its inherent stochasticity. Suchstochasticity is also known as the aleatoric uncertainty and is closely relatedto the notion of risk. In this work, we make the first attempt to studyrisk-sensitive deep reinforcement learning under the average reward settingwith the variance risk criteria. In particular, we focus on avariance-constrained policy optimization problem where the goal is to find apolicy that maximizes the expected value of the long-run average reward,subject to a constraint that the long-run variance of the average reward isupper bounded by a threshold. Utilizing Lagrangian and Fenchel dualities, wetransform the original problem into an unconstrained saddle-point policyoptimization problem, and propose an actor-critic algorithm that iterativelyand efficiently updates the policy, the Lagrange multiplier, and the Fencheldual variable. When both the value and policy functions are represented bymulti-layer overparameterized neural networks, we prove that our actor-criticalgorithm generates a sequence of policies that finds a globally optimal policyat a sublinear rate. Further, We provide numerical studies of the proposedmethod using two real datasets to back up the theoretical results.

Risk-Sensitive Deep RL: Variance-Constrained Actor-Critic Provably Finds Globally Optimal Policy