English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In both the human brain and any general artificial intelligence (AI), arepresentation of the past is necessary to predict the future. However, perfectstorage of all experiences is not feasible. One approach utilized in manyapplications, including reward prediction in reinforcement learning, is toretain recently active features of experience in a buffer. Despite its priorsuccesses, we show that the fixed length buffer renders Deep Q-learningNetworks (DQNs) fragile to changes in the scale over which information can belearned. To enable learning when the relevant temporal scales in theenvironment are not known *a priori*, recent advances in psychology andneuroscience suggest that the brain maintains a compressed representation ofthe past. Here we introduce a neurally-plausible, scale-free memoryrepresentation we call Scale-Invariant Temporal History (SITH) for use withartificial agents. This representation covers an exponentially large period oftime by sacrificing temporal accuracy for events further in the past. Wedemonstrate the utility of this representation by comparing the performance ofagents given SITH, buffer, and exponential decay representations in learning toplay video games at different levels of complexity. In these environments, SITHexhibits better learning performance by storing information for longertimescales than a fixed-size buffer, and representing this information moreclearly than a set of exponentially decayed features. Finally, we discuss howthe application of SITH, along with other human-inspired models of cognition,could improve reinforcement and machine learning algorithms in general.

Scale-invariant temporal history (SITH): optimal slicing of the past in an uncertain world