English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Reinforcement learning (RL) has been successfully used to solve manycontinuous control tasks. Despite its impressive results however, fundamentalquestions regarding the sample complexity of RL on continuous problems remainopen. We study the performance of RL in this setting by considering thebehavior of the Least-Squares Temporal Difference (LSTD) estimator on theclassic Linear Quadratic Regulator (LQR) problem from optimal control. We givethe first finite-time analysis of the number of samples needed to estimate thevalue function for a fixed static state-feedback policy to within$\varepsilon$-relative error. In the process of deriving our result, we give ageneral characterization for when the minimum eigenvalue of the empiricalcovariance matrix formed along the sample path of a fast-mixing stochasticprocess concentrates above zero, extending a result by Koltchinskii andMendelson in the independent covariates setting. Finally, we provideexperimental evidence indicating that our analysis correctly captures thequalitative behavior of LSTD on several LQR instances.

Least-Squares Temporal Difference Learning for the Linear Quadratic Regulator