English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In the past few years, off-policy reinforcement learning methods have shownpromising results in their application for robot control. Deep Q-learning,however, still suffers from poor data-efficiency and is susceptible tostochasticity in the environment or reward functions which is limiting withregard to real-world applications. We alleviate these problems by proposing twonovel off-policy Temporal-Difference formulations: (1) Truncated Q-functionswhich represent the return for the first n steps of a target-policy rolloutw.r.t. the full action-value and (2) Shifted Q-functions, acting as thefarsighted return after this truncated rollout. This decomposition allows us tooptimize both parts with their individual learning rates, achieving significantlearning speedup. We prove that the combination of these short- and long-termpredictions is a representation of the full return, leading to the CompositeQ-learning algorithm. We show the efficacy of Composite Q-learning in thetabular case and compare Deep Composite Q-learning with TD3 and TD3(Delta),which we introduce as an off-policy variant of TD(Delta). Moreover, we showthat Composite TD3 outperforms TD3 as well as state-of-the-art compositionalQ-learning approaches significantly in terms of data-efficiency in multiplesimulated robot tasks and that Composite Q-learning is robust to stochasticenvironments and reward functions.

Composite Q-learning: Multi-scale Q-function Decomposition and Separable Optimization