English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

One of the fundamental challenges for offline reinforcement learning (RL) isensuring robustness to data distribution. Whether the data originates from anear-optimal policy or not, we anticipate that an algorithm should demonstrateits ability to learn an effective control policy that seamlessly aligns withthe inherent distribution of offline data. Unfortunately, behaviorregularization, a simple yet effective offline RL algorithm, tends to strugglein this regard. In this paper, we propose a new algorithm that substantiallyenhances behavior-regularization based on conservative policy iteration. Ourkey observation is that by iteratively refining the reference policy used forbehavior regularization, conservative policy update guarantees graduallyimprovement, while also implicitly avoiding querying out-of-sample actions toprevent catastrophic learning failures. We prove that in the tabular settingthis algorithm is capable of learning the optimal policy covered by the offlinedataset, commonly referred to as the in-sample optimal policy. We then exploreseveral implementation details of the algorithm when function approximationsare applied. The resulting algorithm is easy to implement, requiring only a fewlines of code modification to existing methods. Experimental results on theD4RL benchmark indicate that our method outperforms previous state-of-the-artbaselines in most tasks, clearly demonstrate its superiority over behaviorregularization.

Iteratively Refined Behavior Regularization for Offline Reinforcement Learning