English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many real-world continuous control problems are in the dilemma of weighingthe pros and cons, multi-objective reinforcement learning (MORL) serves as ageneric framework of learning control policies for different preferences overobjectives. However, the existing MORL methods either rely on multiple passesof explicit search for finding the Pareto front and therefore are notsample-efficient, or utilizes a shared policy network for coarse knowledgesharing among policies. To boost the sample efficiency of MORL, we proposeQ-Pensieve, a policy improvement scheme that stores a collection of Q-snapshotsto jointly determine the policy update direction and thereby enables datasharing at the policy level. We show that Q-Pensieve can be naturallyintegrated with soft policy iteration with convergence guarantee. Tosubstantiate this concept, we propose the technique of Q replay buffer, whichstores the learned Q-networks from the past iterations, and arrive at apractical actor-critic implementation. Through extensive experiments and anablation study, we demonstrate that with much fewer samples, the proposedalgorithm can outperform the benchmark MORL methods on a variety of MORLbenchmark tasks.

Q-Pensieve: Boosting Sample Efficiency of Multi-Objective RL Through Memory Sharing of Q-Snapshots