English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Massive practical works addressed by Deep Q-network (DQN) algorithm haveindicated that stochastic policy, despite its simplicity, is the mostfrequently used exploration approach. However, most existing stochasticexploration approaches either explore new actions heuristically regardless ofQ-values or inevitably introduce bias into the learning process to couple thesampling with Q-values. In this paper, we propose a novel preference-guided$\epsilon$-greedy exploration algorithm that can efficiently learn the actiondistribution in line with the landscape of Q-values for DQN without introducingadditional bias. Specifically, we design a dual architecture consisting of twobranches, one of which is a copy of DQN, namely the Q-branch. The other branch,which we call the preference branch, learns the action preference that the DQNimplicit follows. We theoretically prove that the policy improvement theoremholds for the preference-guided $\epsilon$-greedy policy and experimentallyshow that the inferred action preference distribution aligns with the landscapeof corresponding Q-values. Consequently, preference-guided $\epsilon$-greedyexploration motivates the DQN agent to take diverse actions, i.e., actions withlarger Q-values can be sampled more frequently whereas actions with smallerQ-values still have a chance to be explored, thus encouraging the exploration.We assess the proposed method with four well-known DQN variants in ninedifferent environments. Extensive results confirm the superiority of ourproposed method in terms of performance and convergence speed.  Index Terms- Preference-guided exploration, stochastic policy, dataefficiency, deep reinforcement learning, deep Q-learning.

Sampling Efficient Deep Reinforcement Learning through Preference-Guided Stochastic Exploration