English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

A seamless integration of robots into human environments requires robots tolearn how to use existing human tools. Current approaches for learning toolmanipulation skills mostly rely on expert demonstrations provided in the targetrobot environment, for example, by manually guiding the robot manipulator or byteleoperation. In this work, we introduce an automated approach that replacesan expert demonstration with a Youtube video for learning a tool manipulationstrategy. The main contributions are twofold. First, we design an alignmentprocedure that aligns the simulated environment with the real-world sceneobserved in the video. This is formulated as an optimization problem that findsa spatial alignment of the tool trajectory to maximize the sparse goal rewardgiven by the environment. Second, we describe an imitation learning approachthat focuses on the trajectory of the tool rather than the motion of the human.For this we combine reinforcement learning with an optimization procedure tofind a control policy and the placement of the robot based on the tool motionin the aligned environment. We demonstrate the proposed approach on spade,scythe and hammer tools in simulation, and show the effectiveness of thetrained policy for the spade on a real Franka Emika Panda robot demonstration.

Learning to Manipulate Tools by Aligning Simulation to Video Demonstration