English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Our goal is for robots to follow natural language instructions like "put thetowel next to the microwave." But getting large amounts of labeled data, i.e.data that contains demonstrations of tasks labeled with the languageinstruction, is prohibitive. In contrast, obtaining policies that respond toimage goals is much easier, because any autonomous trial or demonstration canbe labeled in hindsight with its final state as the goal. In this work, wecontribute a method that taps into joint image- and goal- conditioned policieswith language using only a small amount of language data. Prior work has madeprogress on this using vision-language models or by jointly traininglanguage-goal-conditioned policies, but so far neither method has scaledeffectively to real-world robot tasks without significant human annotation. Ourmethod achieves robust performance in the real world by learning an embeddingfrom the labeled data that aligns language not to the goal image, but rather tothe desired change between the start and goal images that the instructioncorresponds to. We then train a policy on this embedding: the policy benefitsfrom all the unlabeled data, but the aligned embedding provides an interfacefor language to steer the policy. We show instruction following across avariety of manipulation tasks in different scenes, with generalization tolanguage instructions outside of the labeled data. Videos and code for ourapproach can be found on our website: https://rail-berkeley.github.io/grif/ .

Goal Representations for Instruction Following: A Semi-Supervised Language Interface to Control