English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Model-based reinforcement learning (RL) often achieves higher sampleefficiency in practice than model-free RL by learning a dynamics model togenerate samples for policy learning. Previous works learn a dynamics modelthat fits under the empirical state-action visitation distribution for allhistorical policies, i.e., the sample replay buffer. However, in this paper, weobserve that fitting the dynamics model under the distribution for \emph{allhistorical policies} does not necessarily benefit model prediction for the\emph{current policy} since the policy in use is constantly evolving over time.The evolving policy during training will cause state-action visitationdistribution shifts. We theoretically analyze how this distribution shift overhistorical policies affects the model learning and model rollouts. We thenpropose a novel dynamics model learning method, named \textit{Policy-adaptedDynamics Model Learning (PDML)}. PDML dynamically adjusts the historical policymixture distribution to ensure the learned model can continually adapt to thestate-action visitation distribution of the evolving policy. Experiments on arange of continuous control environments in MuJoCo show that PDML achievessignificant improvement in sample efficiency and higher asymptotic performancecombined with the state-of-the-art model-based RL methods.

Live in the Moment: Learning Dynamics Model Adapted to Evolving Policy