English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Offline reinforcement learning is used to train policies in scenarios wherereal-time access to the environment is expensive or impossible. As a naturalconsequence of these harsh conditions, an agent may lack the resources to fullyobserve the online environment before taking an action. We dub this situationthe resource-constrained setting. This leads to situations where the offlinedataset (available for training) can contain fully processed features (usingpowerful language models, image models, complex sensors, etc.) which are notavailable when actions are actually taken online. This disconnect leads to aninteresting and unexplored problem in offline RL: Is it possible to use arichly processed offline dataset to train a policy which has access to fewerfeatures in the online environment? In this work, we introduce and formalizethis novel resource-constrained problem setting. We highlight the performancegap between policies trained using the full offline dataset and policiestrained using limited features. We address this performance gap with a policytransfer algorithm which first trains a teacher agent using the offline datasetwhere features are fully available, and then transfers this knowledge to astudent agent that only uses the resource-constrained features. To bettercapture the challenge of this setting, we propose a data collection procedure:Resource Constrained-Datasets for RL (RC-D4RL). We evaluate our transferalgorithm on RC-D4RL and the popular D4RL benchmarks and observe consistentimprovement over the baseline (TD3+BC without transfer). The code for theexperiments is available at https://github.com/JayanthRR/RC-OfflineRL.

Offline RL With Resource Constrained Online Deployment