English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Bayesian inverse reinforcement learning (IRL) methods are ideal for safeimitation learning, as they allow a learning agent to reason about rewarduncertainty and the safety of a learned policy. However, Bayesian IRL iscomputationally intractable for high-dimensional problems because each samplefrom the posterior requires solving an entire Markov Decision Process (MDP).While there exist non-Bayesian deep IRL methods, these methods typically inferpoint estimates of reward functions, precluding rigorous safety and uncertaintyanalysis. We propose Bayesian Reward Extrapolation (B-REX), a highly efficient,preference-based Bayesian reward learning algorithm that scales tohigh-dimensional, visual control tasks. Our approach uses successor featurerepresentations and preferences over demonstrations to efficiently generatesamples from the posterior distribution over the demonstrator's reward functionwithout requiring an MDP solver. Using samples from the posterior, wedemonstrate how to calculate high-confidence bounds on policy performance inthe imitation learning setting, in which the ground-truth reward function isunknown. We evaluate our proposed approach on the task of learning to playAtari games via imitation learning from pixel inputs, with no access to thegame score. We demonstrate that B-REX learns imitation policies that arecompetitive with a state-of-the-art deep imitation learning method that onlylearns a point estimate of the reward function. Furthermore, we demonstratethat samples from the posterior generated via B-REX can be used to computehigh-confidence performance bounds for a variety of evaluation policies. Weshow that high-confidence performance bounds are useful for accurately rankingdifferent evaluation policies when the reward function is unknown. We alsodemonstrate that high-confidence performance bounds may be useful for detectingreward hacking.

Deep Bayesian Reward Learning from Preferences