English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reinforcement learning considers the problem of finding policies thatmaximize an expected cumulative reward in a Markov decision process withunknown transition probabilities. In this paper we consider the problem offinding optimal policies assuming that they belong to a reproducing kernelHilbert space (RKHS). To that end we compute unbiased stochastic gradients ofthe value function which we use as ascent directions to update the policy. Amajor drawback of policy gradient-type algorithms is that they are limited toepisodic tasks unless stationarity assumptions are imposed. Hence preventingthese algorithms to be fully implemented online, which is a desirable propertyfor systems that need to adapt to new tasks and/or environments in deployment.The main requirement for a policy gradient algorithm to work is that theestimate of the gradient at any point in time is an ascent direction for theinitial value function. In this work we establish that indeed this is the casewhich enables to show the convergence of the online algorithm to the criticalpoints of the initial value function. A numerical example shows the ability ofour online algorithm to learn to solve a navigation and surveillance problem,in which an agent must loop between to goal locations. This examplecorroborates our theoretical findings about the ascent directions of subsequentstochastic gradients. It also shows how the agent running our online algorithmsucceeds in learning to navigate, following a continuing cyclic trajectory thatdoes not comply with the standard stationarity assumptions in the literaturefor non episodic training.

Policy Gradient for Continuing Tasks in Non-stationary Markov Decision Processes