English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many advances that have improved the robustness and efficiency of deepreinforcement learning (RL) algorithms can, in one way or another, beunderstood as introducing additional objectives or constraints in the policyoptimization step. This includes ideas as far ranging as exploration bonuses,entropy regularization, and regularization toward teachers or data priors.Often, the task reward and auxiliary objectives are in conflict, and in thispaper we argue that this makes it natural to treat these cases as instances ofmulti-objective (MO) optimization problems. We demonstrate how this perspectiveallows us to develop novel and more effective RL algorithms. In particular, wefocus on offline RL and finetuning as case studies, and show that existingapproaches can be understood as MO algorithms relying on linear scalarization.We hypothesize that replacing linear scalarization with a better algorithm canimprove performance. We introduce Distillation of a Mixture of Experts (DiME),a new MORL algorithm that outperforms linear scalarization and can be appliedto these non-standard MO problems. We demonstrate that for offline RL, DiMEleads to a simple new algorithm that outperforms state-of-the-art. Forfinetuning, we derive new algorithms that learn to outperform the teacherpolicy.

On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuning