English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

This paper makes one step forward towards characterizing a new family of\textit{model-free} Deep Reinforcement Learning (DRL) algorithms. The aim ofthese algorithms is to jointly learn an approximation of the state-valuefunction ($V$), alongside an approximation of the state-action value function($Q$). Our analysis starts with a thorough study of the Deep Quality-ValueLearning (DQV) algorithm, a DRL algorithm which has been shown to outperformpopular techniques such as Deep-Q-Learning (DQN) and Double-Deep-Q-Learning(DDQN) \cite{sabatelli2018deep}. Intending to investigate why DQV's learningdynamics allow this algorithm to perform so well, we formulate a set ofresearch questions which help us characterize a new family of DRL algorithms.Among our results, we present some specific cases in which DQV's performancecan get harmed and introduce a novel \textit{off-policy} DRL algorithm, calledDQV-Max, which can outperform DQV. We then study the behavior of the $V$ and$Q$ functions that are learned by DQV and DQV-Max and show that both algorithmsmight perform so well on several DRL test-beds because they are less prone tosuffer from the overestimation bias of the $Q$ function.

Approximating two value functions instead of one: towards characterizing a new family of Deep Reinforcement Learning algorithms