English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Model-based reinforcement learning (MBRL) holds the promise ofsample-efficient learning by utilizing a world model, which models how theenvironment works and typically encompasses components for two tasks:observation modeling and reward modeling. In this paper, through a dedicatedempirical investigation, we gain a deeper understanding of the role each taskplays in world models and uncover the overlooked potential of more efficientMBRL by harmonizing the interference between observation and reward modeling.Our key insight is that while prevalent approaches of explicit MBRL attempt torestore abundant details of the environment through observation models, it isdifficult due to the environment's complexity and limited model capacity. Onthe other hand, reward models, while dominating in implicit MBRL and adept atlearning task-centric dynamics, are inadequate for sample-efficient learningwithout richer learning signals. Capitalizing on these insights anddiscoveries, we propose a simple yet effective method, Harmony World Models(HarmonyWM), that introduces a lightweight harmonizer to maintain a dynamicequilibrium between the two tasks in world model learning. Our experiments onthree visual control domains show that the base MBRL method equipped withHarmonyWM gains 10%-55% absolute performance boosts.

Harmony World Models: Boosting Sample Efficiency for Model-based Reinforcement Learning