English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In spite of the large literature on reinforcement learning (RL) algorithmsfor partially observable Markov decision processes (POMDPs), a completetheoretical understanding is still lacking. In a partially observable setting,the history of data available to the agent increases over time so mostpractical algorithms either truncate the history to a finite window or compressit using a recurrent neural network leading to an agent state that isnon-Markovian. In this paper, it is shown that in spite of the lack of theMarkov property, recurrent Q-learning (RQL) converges in the tabular setting.Moreover, it is shown that the quality of the converged limit depends on thequality of the representation which is quantified in terms of what is known asan approximate information state (AIS). Based on this characterization of theapproximation error, a variant of RQL with AIS losses is presented. Thisvariant performs better than a strong baseline for RQL that does not use AISlosses. It is demonstrated that there is a strong correlation between theperformance of RQL over time and the loss associated with the AISrepresentation.

Approximate information state based convergence analysis of recurrent Q-learning