English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Learning to imitate expert behavior from demonstrations can be challenging,especially in environments with high-dimensional, continuous observations andunknown dynamics. Supervised learning methods based on behavioral cloning (BC)suffer from distribution shift: because the agent greedily imitatesdemonstrated actions, it can drift away from demonstrated states due to erroraccumulation. Recent methods based on reinforcement learning (RL), such asinverse RL and generative adversarial imitation learning (GAIL), overcome thisissue by training an RL agent to match the demonstrations over a long horizon.Since the true reward function for the task is unknown, these methods learn areward function from the demonstrations, often using complex and brittleapproximation techniques that involve adversarial training. We propose a simplealternative that still uses RL, but does not require learning a rewardfunction. The key idea is to provide the agent with an incentive to match thedemonstrations over a long horizon, by encouraging it to return to demonstratedstates upon encountering new, out-of-distribution states. We accomplish this bygiving the agent a constant reward of r=+1 for matching the demonstrated actionin a demonstrated state, and a constant reward of r=0 for all other behavior.Our method, which we call soft Q imitation learning (SQIL), can be implementedwith a handful of minor modifications to any standard Q-learning or off-policyactor-critic algorithm. Theoretically, we show that SQIL can be interpreted asa regularized variant of BC that uses a sparsity prior to encouragelong-horizon imitation. Empirically, we show that SQIL outperforms BC andachieves competitive results compared to GAIL, on a variety of image-based andlow-dimensional tasks in Box2D, Atari, and MuJoCo.

SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards