English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

In preference-based Reinforcement Learning (RL), obtaining a large number ofpreference labels are both time-consuming and costly. Furthermore, the queriedhuman preferences cannot be utilized for the new tasks. In this paper, wepropose Zero-shot Cross-task Preference Alignment and Robust Reward Learning(PEARL), which learns policies from cross-task preference transfer without anyhuman labels of the target task. Our contributions include two novel componentsthat facilitate the transfer and learning process. The first is Cross-taskPreference Alignment (CPA), which transfers the preferences between tasks viaoptimal transport. The key idea of CPA is to use Gromov-Wasserstein distance toalign the trajectories between tasks, and the solved optimal transport matrixserves as the correspondence between trajectories. The target task preferencesare computed as the weighted sum of source task preference labels with thecorrespondence as weights. Moreover, to ensure robust learning from thesetransferred labels, we introduce Robust Reward Learning (RRL), which considersboth reward mean and uncertainty by modeling rewards as Gaussian distributions.Empirical results on robotic manipulation tasks from Meta-World and Robomimicdemonstrate that our method is capable of transferring preference labels acrosstasks accurately and then learns well-behaved policies. Notably, our approachsignificantly exceeds existing methods when there are few human preferences.The code and videos of our method are available at:https://sites.google.com/view/pearl-preference.

PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation