English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider the discrete-time infinite-horizon optimal control problemformalized by Markov Decision Processes. We revisit the work of Bertsekas andIoffe, that introduced $\lambda$ Policy Iteration, a family of algorithmsparameterized by $\lambda$ that generalizes the standard algorithms ValueIteration and Policy Iteration, and has some deep connections with the TemporalDifferences algorithm TD($\lambda$) described by Sutton and Barto. We deepenthe original theory developped by the authors by providing convergence ratebounds which generalize standard bounds for Value Iteration described forinstance by Puterman. Then, the main contribution of this paper is to developthe theory of this algorithm when it is used in an approximate form and showthat this is sound. Doing so, we extend and unify the separate analysesdevelopped by Munos for Approximate Value Iteration and Approximate PolicyIteration. Eventually, we revisit the use of this algorithm in the training ofa Tetris playing controller as originally done by Bertsekas and Ioffe. Weprovide an original performance bound that can be applied to such anundiscounted control problem. Our empirical results are different from those ofBertsekas and Ioffe (which were originally qualified as "paradoxical" and"intriguing"), and much more conform to what one would expect from a learningexperiment. We discuss the possible reason for such a difference.

Performance Bounds for Lambda Policy Iteration and Application to the Game of Tetris