English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Reinforcement Learning (RL) can model complex behavior policies forgoal-directed sequential decision making tasks. A hallmark of RL algorithms isTemporal Difference (TD) learning: value function for the current state ismoved towards a bootstrapped target that is estimated using next state's valuefunction. $\lambda$-returns generalize beyond 1-step returns and strike abalance between Monte Carlo and TD learning methods. While lambda-returns havebeen extensively studied in RL, they haven't been explored a lot in Deep RL.This paper's first contribution is an exhaustive benchmarking oflambda-returns. Although mathematically tractable, the use of exponentiallydecaying weighting of n-step returns based targets in lambda-returns is arather ad-hoc design choice. Our second major contribution is that we propose ageneralization of lambda-returns called Confidence-based Autodidactic Returns(CAR), wherein the RL agent learns the weighting of the n-step returns in anend-to-end manner. This allows the agent to learn to decide how much it wantsto weigh the n-step returns based targets. In contrast, lambda-returns restrictRL agents to use an exponentially decaying weighting scheme. Autodidacticreturns can be used for improving any RL algorithm which uses TD learning. Weempirically demonstrate that using sophisticated weighted mixtures ofmulti-step returns (like CAR and lambda-returns) considerably outperforms theuse of n-step returns. We perform our experiments on the Asynchronous AdvantageActor Critic (A3C) algorithm in the Atari 2600 domain.

Learning to Mix n-Step Returns: Generalizing lambda-Returns for Deep Reinforcement Learning