English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Model-based offline reinforcement learning (RL) aims to find highly rewardingpolicy, by leveraging a previously collected static dataset and a dynamicsmodel. While the dynamics model learned through reuse of the static dataset,its generalization ability hopefully promotes policy learning if properlyutilized. To that end, several works propose to quantify the uncertainty ofpredicted dynamics, and explicitly apply it to penalize reward. However, as thedynamics and the reward are intrinsically different factors in context of MDP,characterizing the impact of dynamics uncertainty through reward penalty mayincur unexpected tradeoff between model utilization and risk avoidance. In thiswork, we instead maintain a belief distribution over dynamics, andevaluate/optimize policy through biased sampling from the belief. The samplingprocedure, biased towards pessimism, is derived based on an alternating Markovgame formulation of offline RL. We formally show that the biased samplingnaturally induces an updated dynamics belief with policy-dependent reweightingfactor, termed Pessimism-Modulated Dynamics Belief. To improve policy, wedevise an iterative regularized policy optimization algorithm for the game,with guarantee of monotonous improvement under certain condition. To makepractical, we further devise an offline RL algorithm to approximately find thesolution. Empirical results show that the proposed approach achievesstate-of-the-art performance on a wide range of benchmark tasks.

Model-Based Offline Reinforcement Learning with Pessimism-Modulated Dynamics Belief