English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent works have shown that Large Language Models (LLMs) can be applied toground natural language to a wide variety of robot skills. However, inpractice, learning multi-task, language-conditioned robotic skills typicallyrequires large-scale data collection and frequent human intervention to resetthe environment or help correcting the current policies. In this work, wepropose a novel approach to efficiently learn general-purposelanguage-conditioned robot skills from unstructured, offline and reset-freedata in the real world by exploiting a self-supervised visuo-lingual affordancemodel, which requires annotating as little as 1% of the total data withlanguage. We evaluate our method in extensive experiments both in simulated andreal-world robotic tasks, achieving state-of-the-art performance on thechallenging CALVIN benchmark and learning over 25 distinct visuomotormanipulation tasks with a single policy in the real world. We find that whenpaired with LLMs to break down abstract natural language instructions intosubgoals via few-shot prompting, our method is capable of completinglong-horizon, multi-tier tasks in the real world, while requiring an order ofmagnitude less data than previous approaches. Code and videos are available athttp://hulc2.cs.uni-freiburg.de

Grounding Language with Visual Affordances over Unstructured Data