English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The field of visual representation learning has seen explosive growth in thepast years, but its benefits in robotics have been surprisingly limited so far.Prior work uses generic visual representations as a basis to learn(task-specific) robot action policies (e.g., via behavior cloning). While thevisual representations do accelerate learning, they are primarily used toencode visual observations. Thus, action information has to be derived purelyfrom robot data, which is expensive to collect! In this work, we present ascalable alternative where the visual representations can help directly inferrobot actions. We observe that vision encoders express relationships betweenimage observations as distances (e.g., via embedding dot product) that could beused to efficiently plan robot behavior. We operationalize this insight anddevelop a simple algorithm for acquiring a distance function and dynamicspredictor, by fine-tuning a pre-trained representation on human collected videosequences. The final method is able to substantially outperform traditionalrobot learning baselines (e.g., 70% success v.s. 50% for behavior cloning onpick-place) on a suite of diverse real-world manipulation tasks. It can alsogeneralize to novel objects, without using any robot demonstrations duringtrain time. For visualizations of the learned policies please check:https://agi-labs.github.io/manipulate-by-seeing/.

Manipulate by Seeing: Creating Manipulation Controllers from Pre-Trained Representations