English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, a sparse Markov decision process (MDP) with novel causalsparse Tsallis entropy regularization is proposed.The proposed policyregularization induces a sparse and multi-modal optimal policy distribution ofa sparse MDP. The full mathematical analysis of the proposed sparse MDP isprovided.We first analyze the optimality condition of a sparse MDP. Then, wepropose a sparse value iteration method which solves a sparse MDP and thenprove the convergence and optimality of sparse value iteration using the Banachfixed point theorem. The proposed sparse MDP is compared to soft MDPs whichutilize causal entropy regularization. We show that the performance error of asparse MDP has a constant bound, while the error of a soft MDP increaseslogarithmically with respect to the number of actions, where this performanceerror is caused by the introduced regularization term. In experiments, we applysparse MDPs to reinforcement learning problems. The proposed method outperformsexisting methods in terms of the convergence speed and performance.

Sparse Markov Decision Processes with Causal Sparse Tsallis Entropy Regularization for Reinforcement Learning