English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent methods for imitation learning directly learn a $Q$-function using animplicit reward formulation rather than an explicit reward function. However,these methods generally require implicit reward regularization to improvestability and often mistreat absorbing states. Previous works show that asquared norm regularization on the implicit reward function is effective, butdo not provide a theoretical analysis of the resulting properties of thealgorithms. In this work, we show that using this regularizer under a mixturedistribution of the policy and the expert provides a particularly illuminatingperspective: the original objective can be understood as squared Bellman errorminimization, and the corresponding optimization problem minimizes a bounded$\chi^2$-Divergence between the expert and the mixture distribution. Thisperspective allows us to address instabilities and properly treat absorbingstates. We show that our method, Least Squares Inverse Q-Learning (LS-IQ),outperforms state-of-the-art algorithms, particularly in environments withabsorbing states. Finally, we propose to use an inverse dynamics model to learnfrom observations only. Using this approach, we retain performance in settingswhere no expert actions are available.

LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learning