English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Current state-of-the-art model-based reinforcement learning algorithms usetrajectory sampling methods, such as the Cross-Entropy Method (CEM), forplanning in continuous control settings. These zeroth-order optimizers requiresampling a large number of trajectory rollouts to select an optimal action,which scales poorly for large prediction horizons or high dimensional actionspaces. First-order methods that use the gradients of the rewards with respectto the actions as an update can mitigate this issue, but suffer from localoptima due to the non-convex optimization landscape. To overcome these issuesand achieve the best of both worlds, we propose a novel planner, Cross-EntropyMethod with Gradient Descent (CEM-GD), that combines first-order methods withCEM. At the beginning of execution, CEM-GD uses CEM to sample a significantamount of trajectory rollouts to explore the optimization landscape and avoidpoor local minima. It then uses the top trajectories as initialization forgradient descent and applies gradient updates to each of these trajectories tofind the optimal action sequence. At each subsequent time step, however, CEM-GDsamples much fewer trajectories from CEM before applying gradient updates. Weshow that as the dimensionality of the planning problem increases, CEM-GDmaintains desirable performance with a constant small number of samples byusing the gradient information, while avoiding local optima using initiallywell-sampled trajectories. Furthermore, CEM-GD achieves better performance thanCEM on a variety of continuous control benchmarks in MuJoCo with 100x fewersamples per time step, resulting in around 25% less computation time and 10%less memory usage. The implementation of CEM-GD is available at$\href{https://github.com/KevinHuang8/CEM-GD}{\text{https://github.com/KevinHuang8/CEM-GD}}$.

CEM-GD: Cross-Entropy Method with Gradient Descent Planner for Model-Based Reinforcement Learning