English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Dyna-style model-based reinforcement learning contains two phases: modelrollouts to generate sample for policy learning and real environmentexploration using current policy for dynamics model learning. However, due tothe complex real-world environment, it is inevitable to learn an imperfectdynamics model with model prediction error, which can further mislead policylearning and result in sub-optimal solutions. In this paper, we propose$\texttt{COPlanner}$, a planning-driven framework for model-based methods toaddress the inaccurately learned dynamics model problem with conservative modelrollouts and optimistic environment exploration. $\texttt{COPlanner}$ leveragesan uncertainty-aware policy-guided model predictive control (UP-MPC) componentto plan for multi-step uncertainty estimation. This estimated uncertainty thenserves as a penalty during model rollouts and as a bonus during realenvironment exploration respectively, to choose actions. Consequently,$\texttt{COPlanner}$ can avoid model uncertain regions through conservativemodel rollouts, thereby alleviating the influence of model error.Simultaneously, it explores high-reward model uncertain regions to reduce modelerror actively through optimistic real environment exploration.$\texttt{COPlanner}$ is a plug-and-play framework that can be applied to anydyna-style model-based methods. Experimental results on a series ofproprioceptive and visual continuous control tasks demonstrate that both sampleefficiency and asymptotic performance of strong model-based methods aresignificantly improved combined with $\texttt{COPlanner}$.

COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL