English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Our goal is for agents to optimize the right reward function, despite howdifficult it is for us to specify what that is. Inverse Reinforcement Learning(IRL) enables us to infer reward functions from demonstrations, but it usuallyassumes that the expert is noisily optimal. Real people, on the other hand,often have systematic biases: risk-aversion, myopia, etc. One option is to tryto characterize these biases and account for them explicitly during learning.But in the era of deep learning, a natural suggestion researchers make is toavoid mathematical models of human behavior that are fraught with specificassumptions, and instead use a purely data-driven approach. We decided to putthis to the test -- rather than relying on assumptions about which specificbias the demonstrator has when planning, we instead learn the demonstrator'splanning algorithm that they use to generate demonstrations, as adifferentiable planner. Our exploration yielded mixed findings: on the onehand, learning the planner can lead to better reward inference than relying onthe wrong assumption; on the other hand, this benefit is dwarfed by the loss weincur by going from an exact to a differentiable planner. This suggest that atleast for the foreseeable future, agents need a middle ground between theflexibility of data-driven methods and the useful bias of known human biases.Code is available at https://tinyurl.com/learningbiases.

On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference