English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We propose a stochastic approximation (SA) based method with randomization ofsamples for policy evaluation using the least squares temporal difference(LSTD) algorithm. Our proposed scheme is equivalent to running regular temporaldifference learning with linear function approximation, albeit with samplespicked uniformly from a given dataset. Our method results in an $O(d)$improvement in complexity in comparison to LSTD, where $d$ is the dimension ofthe data. We provide non-asymptotic bounds for our proposed method, both inhigh probability and in expectation, under the assumption that the matrixunderlying the LSTD solution is positive definite. The latter assumption can beeasily satisfied for the pathwise LSTD variant proposed in [23]. Moreover, wealso establish that using our method in place of LSTD does not impact the rateof convergence of the approximate value function to the true value function.These rate results coupled with the low computational complexity of our methodmake it attractive for implementation in big data settings, where $d$ is large.A similar low-complexity alternative for least squares regression is well-knownas the stochastic gradient descent (SGD) algorithm. We provide finite-timebounds for SGD. We demonstrate the practicality of our method as an efficientalternative for pathwise LSTD empirically by combining it with the leastsquares policy iteration (LSPI) algorithm in a traffic signal controlapplication. We also conduct another set of experiments that combines the SAbased low-complexity variant for least squares regression with the LinUCBalgorithm for contextual bandits, using the large scale news recommendationdataset from Yahoo.

Concentration bounds for temporal difference learning with linear function approximation: The case of batch data and uniform sampling