English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

An inherent problem of reinforcement learning is performing exploration of anenvironment through random actions, of which a large portion can beunproductive. Instead, exploration can be improved by initializing the learningpolicy with an existing (previously learned or hard-coded) oracle policy,offline data, or demonstrations. In the case of using an oracle policy, it canbe unclear how best to incorporate the oracle policy's experience into thelearning policy in a way that maximizes learning sample efficiency. In thispaper, we propose a method termed Critic Confidence Guided Exploration (CCGE)for incorporating such an oracle policy into standard actor-criticreinforcement learning algorithms. More specifically, CCGE takes in the oraclepolicy's actions as suggestions and incorporates this information into thelearning scheme when uncertainty is high, while ignoring it when theuncertainty is low. CCGE is agnostic to methods of estimating uncertainty, andwe show that it is equally effective with two different techniques.Empirically, we evaluate the effect of CCGE on various benchmark reinforcementlearning tasks, and show that this idea can lead to improved sample efficiencyand final performance. Furthermore, when evaluated on sparse rewardenvironments, CCGE is able to perform competitively against adjacent algorithmsthat also leverage an oracle policy. Our experiments show that it is possibleto utilize uncertainty as a heuristic to guide exploration using an oracle inreinforcement learning. We expect that this will inspire more research in thisdirection, where various heuristics are used to determine the direction ofguidance provided to learning.

Some Supervision Required: Incorporating Oracle Policies in Reinforcement Learning via Epistemic Uncertainty Metrics