English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The pre-train and fine-tune paradigm in machine learning has had dramaticsuccess in a wide range of domains because the use of existing data orpre-trained models on the internet enables quick and easy learning of newtasks. We aim to enable this paradigm in robotic reinforcement learning,allowing a robot to learn a new task with little human effort by leveragingdata and models from the Internet. However, reinforcement learning oftenrequires significant human effort in the form of manual reward specification orenvironment resets, even if the policy is pre-trained. We introduce RoboFuME, areset-free fine-tuning system that pre-trains a multi-task manipulation policyfrom diverse datasets of prior experiences and self-improves online to learn atarget task with minimal human intervention. Our insights are to utilizecalibrated offline reinforcement learning techniques to ensure efficient onlinefine-tuning of a pre-trained policy in the presence of distribution shifts andleverage pre-trained vision language models (VLMs) to build a robust rewardclassifier for autonomously providing reward signals during the onlinefine-tuning process. In a diverse set of five real robot manipulation tasks, weshow that our method can incorporate data from an existing robot datasetcollected at a different institution and improve on a target task within aslittle as 3 hours of autonomous real-world experience. We also demonstrate insimulation experiments that our method outperforms prior works that usedifferent RL algorithms or different approaches for predicting rewards. Projectwebsite: https://robofume.github.io

Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning