English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reinforcement learning aims to learn optimal policies from interaction withenvironments whose dynamics are unknown. Many methods rely on the approximationof a value function to derive near-optimal policies. In partially observableenvironments, these functions depend on the complete sequence of observationsand past actions, called the history. In this work, we show empirically thatrecurrent neural networks trained to approximate such value functionsinternally filter the posterior probability distribution of the current stategiven the history, called the belief. More precisely, we show that, as arecurrent neural network learns the Q-function, its hidden states become moreand more correlated with the beliefs of state variables that are relevant tooptimal control. This correlation is measured through their mutual information.In addition, we show that the expected return of an agent increases with theability of its recurrent architecture to reach a high mutual informationbetween its hidden states and the beliefs. Finally, we show that the mutualinformation between the hidden states and the beliefs of variables that areirrelevant for optimal control decreases through the learning process. Insummary, this work shows that in its hidden states, a recurrent neural networkapproximating the Q-function of a partially observable environment reproduces asufficient statistic from the history that is correlated to the relevant partof the belief for taking optimal actions.

Recurrent networks, hidden states and beliefs in partially observable environments