English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The recent success of supervised learning methods on ever larger offlinedatasets has spurred interest in the reinforcement learning (RL) field toinvestigate whether the same paradigms can be translated to RL algorithms. Thisresearch area, known as offline RL, has largely focused on offline policyoptimization, aiming to find a return-maximizing policy exclusively fromoffline data. In this paper, we consider a slightly different approach toincorporating offline data into sequential decision-making. We aim to answerthe question, what unsupervised objectives applied to offline datasets are ableto learn state representations which elevate performance on downstream tasks,whether those downstream tasks be online RL, imitation learning from expertdemonstrations, or even offline policy optimization based on the same offlinedataset? Through a variety of experiments utilizing standard offline RLdatasets, we find that the use of pretraining with unsupervised learningobjectives can dramatically improve the performance of policy learningalgorithms that otherwise yield mediocre performance on their own. Extensiveablations further provide insights into what components of these unsupervisedobjectives -- e.g., reward prediction, continuous or discrete representations,pretraining or finetuning -- are most important and in which settings.

Representation Matters: Offline Pretraining for Sequential Decision Making