English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Deep Q-Networks algorithm (DQN) was the first reinforcement learningalgorithm using deep neural network to successfully surpass human levelperformance in a number of Atari learning environments. However, divergent andunstable behaviour have been long standing issues in DQNs. The unstablebehaviour is often characterised by overestimation in the $Q$-values, commonlyreferred to as the overestimation bias. To address the overestimation bias andthe divergent behaviour, a number of heuristic extensions have been proposed.Notably, multi-step updates have been shown to drastically reduce unstablebehaviour while improving agent's training performance. However, agents areoften highly sensitive to the selection of the multi-step update horizon ($n$),and our empirical experiments show that a poorly chosen static value for $n$can in many cases lead to worse performance than single-step DQN. Inspired bythe success of $n$-step DQN and the effects that multi-step updates have onoverestimation bias, this paper proposes a new algorithm that we call `ElasticStep DQN' (ES-DQN). It dynamically varies the step size horizon in multi-stepupdates based on the similarity of states visited. Our empirical evaluationshows that ES-DQN out-performs $n$-step with fixed $n$ updates, Double DQN andAverage DQN in several OpenAI Gym environments while at the same timealleviating the overestimation bias.

Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks