English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this work, we study the simple yet universally applicable case of rewardshaping in value-based Deep Reinforcement Learning (DRL). We show that rewardshifting in the form of the linear transformation is equivalent to changing theinitialization of the $Q$-function in function approximation. Based on such anequivalence, we bring the key insight that a positive reward shifting leads toconservative exploitation, while a negative reward shifting leads tocuriosity-driven exploration. Accordingly, conservative exploitation improvesoffline RL value estimation, and optimistic value estimation improvesexploration for online RL. We validate our insight on a range of RL tasks andshow its improvement over baselines: (1) In offline RL, the conservativeexploitation leads to improved performance based on off-the-shelf algorithms;(2) In online continuous control, multiple value functions with differentshifting constants can be used to tackle the exploration-exploitation dilemmafor better sample efficiency; (3) In discrete control tasks, a negative rewardshifting yields an improvement over the curiosity-based exploration method.

Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping