English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider Markov Decision Problems defined over continuous state and actionspaces, where an autonomous agent seeks to learn a map from its states toactions so as to maximize its long-term discounted accumulation of rewards. Weaddress this problem by considering Bellman's optimality equation defined overaction-value functions, which we reformulate into a nested non-convexstochastic optimization problem defined over a Reproducing Kernel Hilbert Space(RKHS). We develop a functional generalization of stochastic quasi-gradientmethod to solve it, which, owing to the structure of the RKHS, admits aparameterization in terms of scalar weights and past state-action pairs whichgrows proportionately with the algorithm iteration index. To ameliorate thiscomplexity explosion, we apply Kernel Orthogonal Matching Pursuit to thesequence of kernel weights and dictionaries, which yields a controllable errorin the descent direction of the underlying optimization method. We prove thatthe resulting algorithm, called KQ-Learning, converges with probability 1 to astationary point of this problem, yielding a fixed point of the Bellmanoptimality operator under the hypothesis that it belongs to the RKHS. Underconstant learning rates, we further obtain convergence to a small Bellman errorthat depends on the chosen learning rates. Numerical evaluation on theContinuous Mountain Car and Inverted Pendulum tasks yields convergentparsimonious learned action-value functions, policies that are competitive withthe state of the art, and exhibit reliable, reproducible learning behavior.

Nonparametric Stochastic Compositional Gradient Descent for Q-Learning in Continuous Markov Decision Problems