English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Greedy-GQ is an off-policy two timescale algorithm for optimal control inreinforcement learning. This paper develops the first finite-sample analysisfor the Greedy-GQ algorithm with linear function approximation under Markoviannoise. Our finite-sample analysis provides theoretical justification forchoosing stepsizes for this two timescale algorithm for faster convergence inpractice, and suggests a trade-off between the convergence rate and the qualityof the obtained policy. Our paper extends the finite-sample analyses of twotimescale reinforcement learning algorithms from policy evaluation to optimalcontrol, which is of more practical interest. Specifically, in contrast toexisting finite-sample analyses for two timescale methods, e.g., GTD, GTD2 andTDC, where their objective functions are convex, the objective function of theGreedy-GQ algorithm is non-convex. Moreover, the Greedy-GQ algorithm is alsonot a linear two-timescale stochastic approximation algorithm. Our techniquesin this paper provide a general framework for finite-sample analysis ofnon-convex value-based reinforcement learning algorithms for optimal control.

Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise