English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

This paper investigates training better visual world models for robotmanipulation, i.e., models that can predict future visual observations byconditioning on past frames and robot actions. Specifically, we consider worldmodels that operate on RGB-D frames (RGB-D world models). As opposed tocanonical approaches that handle dynamics prediction mostly implicitly andreconcile it with visual rendering in a single model, we introduce FlowDreamer,which adopts 3D scene flow as explicit motion representations. FlowDreamerfirst predicts 3D scene flow from past frame and action conditions with aU-Net, and then a diffusion model will predict the future frame utilizing thescene flow. FlowDreamer is trained end-to-end despite its modularized nature.We conduct experiments on 4 different benchmarks, covering both videoprediction and visual planning tasks. The results demonstrate that FlowDreamerachieves better performance compared to other baseline RGB-D world models by 7%on semantic similarity, 11% on pixel quality, and 6% on success rate in variousrobot manipulation domains.

FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation