English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

World models are a fundamental component in model-based reinforcementlearning (MBRL). To perform temporally extended and consistent simulations ofthe future in partially observable environments, world models need to possesslong-term memory. However, state-of-the-art MBRL agents, such as Dreamer,predominantly employ recurrent neural networks (RNNs) as their world modelbackbone, which have limited memory capacity. In this paper, we seek to explorealternative world model backbones for improving long-term memory. Inparticular, we investigate the effectiveness of Transformers and StructuredState Space Sequence (S4) models, motivated by their remarkable ability tocapture long-range dependencies in low-dimensional sequences and theircomplementary strengths. We propose S4WM, the first world model compatible withparallelizable SSMs including S4 and its variants. By incorporating latentvariable modeling, S4WM can efficiently generate high-dimensional imagesequences through latent imagination. Furthermore, we extensively compare RNN-,Transformer-, and S4-based world models across four sets of environments, whichwe have tailored to assess crucial memory capabilities of world models,including long-term imagination, context-dependent recall, reward prediction,and memory-based reasoning. Our findings demonstrate that S4WM outperformsTransformer-based world models in terms of long-term memory, while exhibitinggreater efficiency during training and imagination. These results pave the wayfor the development of stronger MBRL agents.

Facing Off World Model Backbones: RNNs, Transformers, and S4