English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The goal of Unsupervised Reinforcement Learning (URL) is to find areward-agnostic prior policy on a task domain, such that the sample-efficiencyon supervised downstream tasks is improved. Although agents initialized withsuch a prior policy can achieve a significantly higher reward with fewersamples when finetuned on the downstream task, it is still an open question howan optimal pretrained prior policy can be achieved in practice. In this work,we present POLTER (Policy Trajectory Ensemble Regularization) - a generalmethod to regularize the pretraining that can be applied to any URL algorithmand is especially useful on data- and knowledge-based URL algorithms. Itutilizes an ensemble of policies that are discovered during pretraining andmoves the policy of the URL algorithm closer to its optimal prior. Our methodis based on a theoretical framework, and we analyze its practical effects on awhite-box benchmark, allowing us to study POLTER with full control. In our mainexperiments, we evaluate POLTER on the Unsupervised Reinforcement LearningBenchmark (URLB), which consists of 12 tasks in 3 domains. We demonstrate thegenerality of our approach by improving the performance of a diverse set ofdata- and knowledge-based URL algorithms by 19% on average and up to 40% in thebest case. Under a fair comparison with tuned baselines and tuned POLTER, weestablish a new state-of-the-art for model-free methods on the URLB.

POLTER: Policy Trajectory Ensemble Regularization for Unsupervised Reinforcement Learning