English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we present a new class of Markov decision processes (MDPs),called Tsallis MDPs, with Tsallis entropy maximization, which generalizesexisting maximum entropy reinforcement learning (RL). A Tsallis MDP provides aunified framework for the original RL problem and RL with various types ofentropy, including the well-known standard Shannon-Gibbs (SG) entropy, using anadditional real-valued parameter, called an entropic index. By controlling theentropic index, we can generate various types of entropy, including the SGentropy, and a different entropy results in a different class of the optimalpolicy in Tsallis MDPs. We also provide a full mathematical analysis of TsallisMDPs, including the optimality condition, performance error bounds, andconvergence. Our theoretical result enables us to use any positive entropicindex in RL. To handle complex and large-scale problems, we propose amodel-free actor-critic RL method using Tsallis entropy maximization. Weevaluate the regularization effect of the Tsallis entropy with various valuesof entropic indices and show that the entropic index controls the explorationtendency of the proposed method. For a different type of RL problems, we findthat a different value of the entropic index is desirable. The proposed methodis evaluated using the MuJoCo simulator and achieves the state-of-the-artperformance.

Tsallis Reinforcement Learning: A Unified Framework for Maximum Entropy Reinforcement Learning