English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Consider the problem of exploration in sparse-reward or reward-freeenvironments, such as in Montezuma's Revenge. In the curiosity-driven paradigm,the agent is rewarded for how much each realized outcome differs from theirpredicted outcome. But using predictive error as intrinsic motivation isfragile in stochastic environments, as the agent may become trapped byhigh-entropy areas of the state-action space, such as a "noisy TV". In thiswork, we study a natural solution derived from structural causal models of theworld: Our key idea is to learn representations of the future that captureprecisely the unpredictable aspects of each outcome -- which we use asadditional input for predictions, such that intrinsic rewards only reflect thepredictable aspects of world dynamics. First, we propose incorporating suchhindsight representations into models to disentangle "noise" from "novelty",yielding Curiosity in Hindsight: a simple and scalable generalization ofcuriosity that is robust to stochasticity. Second, we instantiate thisframework for the recently introduced BYOL-Explore algorithm as our primeexample, resulting in the noise-robust BYOL-Hindsight. Third, we illustrate itsbehavior under a variety of different stochasticities in a grid world, and findimprovements over BYOL-Explore in hard-exploration Atari games with stickyactions. Notably, we show state-of-the-art results in exploring Montezuma'sRevenge with sticky actions, while preserving performance in the non-stickysetting.

Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments