English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

An appropriate reward function is of paramount importance in specifying atask in reinforcement learning (RL). Yet, it is known to be extremelychallenging in practice to design a correct reward function for even simpletasks. Human-in-the-loop (HiL) RL allows humans to communicate complex goals tothe RL agent by providing various types of feedback. However, despite achievinggreat empirical successes, HiL RL usually requires too much feedback from ahuman teacher and also suffers from insufficient theoretical understanding. Inthis paper, we focus on addressing this issue from a theoretical perspective,aiming to provide provably feedback-efficient algorithmic frameworks that takehuman-in-the-loop to specify rewards of given tasks. We provide anactive-learning-based RL algorithm that first explores the environment withoutspecifying a reward function and then asks a human teacher for only a fewqueries about the rewards of a task at some state-action pairs. After that, thealgorithm guarantees to provide a nearly optimal policy for the task with highprobability. We show that, even with the presence of random noise in thefeedback, the algorithm only takes $\widetilde{O}(H{{\dim_{R}^2}})$ queries onthe reward function to provide an $\epsilon$-optimal policy for any $\epsilon >0$. Here $H$ is the horizon of the RL environment, and $\dim_{R}$ specifies thecomplexity of the function class representing the reward function. In contrast,standard RL algorithms require to query the reward function for at least$\Omega(\operatorname{poly}(d, 1/\epsilon))$ state-action pairs where $d$depends on the complexity of the environmental transition.

Provably Feedback-Efficient Reinforcement Learning via Active Reward Learning