English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Human hands possess remarkable dexterity and have long served as a source ofinspiration for robotic manipulation. In this work, we propose a human$\textbf{H}$and$\textbf{-In}$formed visual representation learning framework tosolve difficult $\textbf{Dex}$terous manipulation tasks ($\textbf{H-InDex}$)with reinforcement learning. Our framework consists of three stages: (i)pre-training representations with 3D human hand pose estimation, (ii) offlineadapting representations with self-supervised keypoint detection, and (iii)reinforcement learning with exponential moving average BatchNorm. The last twostages only modify $0.36\%$ parameters of the pre-trained representation intotal, ensuring the knowledge from pre-training is maintained to the fullextent. We empirically study 12 challenging dexterous manipulation tasks andfind that H-InDex largely surpasses strong baseline methods and the recentvisual foundation models for motor control. Code is available athttps://yanjieze.com/H-InDex .

H-InDex: Visual Reinforcement Learning with Hand-Informed Representations for Dexterous Manipulation