English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Exploration bonus derived from the novelty of the states in an environmenthas become a popular approach to motivate exploration for deep reinforcementlearning agents in the past few years. Recent methods such as curiosity-drivenexploration usually estimate the novelty of new observations by the predictionerrors of their system dynamics models. Due to the capacity limitation of themodels and difficulty of performing next-frame prediction, however, thesemethods typically fail to balance between exploration and exploitation inhigh-dimensional observation tasks, resulting in the agents forgetting thevisited paths and exploring those states repeatedly. Such inefficientexploration behavior causes significant performance drops, especially in largeenvironments with sparse reward signals. In this paper, we propose to introducethe concept of optical flow estimation from the field of computer vision todeal with the above issue. We propose to employ optical flow estimation errorsto examine the novelty of new observations, such that agents are able tomemorize and understand the visited states in a more comprehensive fashion. Wecompare our method against the previous approaches in a number of experimentalexperiments. Our results indicate that the proposed method appears to deliversuperior and long-lasting performance than the previous methods. We furtherprovide a set of comprehensive ablative analysis of the proposed method, andinvestigate the impact of optical flow estimation on the learning curves of theDRL agents.

Never Forget: Balancing Exploration and Exploitation via Learning Optical Flow