English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradient(HT-PSG) algorithm to deal with the challenges of sparse rewards in continuouscontrol problems. Sparse reward is common in continuous control robotics taskssuch as manipulation and navigation, and makes the learning problem hard due tonon-trivial estimation of value functions over the state space. This demandseither reward shaping or expert demonstrations for the sparse rewardenvironment. However, obtaining high-quality demonstrations is quite expensiveand sometimes even impossible. We propose a heavy-tailed policy parametrizationalong with a modified momentum-based policy gradient tracking scheme (HT-SPG)to induce a stable exploratory behavior to the algorithm. The proposedalgorithm does not require access to expert demonstrations. We test theperformance of HT-SPG on various benchmark tasks of continuous control withsparse rewards such as 1D Mario, Pathological Mountain Car, Sparse Pendulum inOpenAI Gym, and Sparse MuJoCo environments (Hopper-v2). We show consistentperformance improvement across all tasks in terms of high average cumulativereward. HT-SPG also demonstrates improved convergence speed with minimumsamples, thereby emphasizing the sample efficiency of our proposed algorithm.

Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies