English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We study the estimation of risk-sensitive policies in reinforcement learningproblems defined by a Markov Decision Process (MDPs) whose state and actionspaces are countably finite. Prior efforts are predominately afflicted bycomputational challenges associated with the fact that risk-sensitive MDPs aretime-inconsistent. To ameliorate this issue, we propose a new definition ofrisk, which we call caution, as a penalty function added to the dual objectiveof the linear programming (LP) formulation of reinforcement learning. Thecaution measures the distributional risk of a policy, which is a function ofthe policy's long-term state occupancy distribution. To solve this problem inan online model-free manner, we propose a stochastic variant of primal-dualmethod that uses Kullback-Lieber (KL) divergence as its proximal term. Weestablish that the number of iterations/samples required to attainapproximately optimal solutions of this scheme matches tight dependencies onthe cardinality of the state and action spaces, but differs in its dependenceon the infinity norm of the gradient of the risk measure. Experimentsdemonstrate the merits of this approach for improving the reliability of rewardaccumulation without additional computational burdens.

Cautious Reinforcement Learning via Distributional Risk in the Dual Domain