Open AccessBias Adaptive Statistical Inference Learning Agents for Learning from Human Feedback
Author(s) -
Jonathan I Watson
Publication year - 2021
Publication title -
proceedings of the ... international florida artificial intelligence research society conference
Language(s) - English
Resource type - Journals
eISSN - 2334-0762
pISSN - 2334-0754
DOI - 10.32473/flairs.v34i1.128471
Subject(s) - computer science , oracle , inference , distortion (music) , artificial intelligence , heuristic , parametric statistics , value (mathematics) , signal (programming language) , machine learning , algorithm , mathematics , statistics , amplifier , computer network , software engineering , bandwidth (computing) , programming language
We present a novel technique for learning behaviors from ahuman provided feedback signal that is distorted by system-atic bias. Our technique, which we refer to as BASIL, modelsthe feedback signal as being separable into a heuristic evalu-ation of the utility of an action and a bias value that is drawnfrom a parametric distribution probabilistically, where thedistribution is defined by unknown parameters. We presentthe general form of the technique as well as a specific algo-rithm for integrating the technique with the TAMER algo-rithm for bias values drawn from a normal distribution. Wetest our algorithm against standard TAMER in the domain ofTetris using a synthetic oracle that provides feedback undervarying levels of distortion. We find our algorithm can learnvery quickly under bias distortions that entirely stymie thelearning of classic TAMER.