English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Motivated by the emerging use of multi-agent reinforcement learning (MARL) inengineering applications such as networked robotics, swarming drones, andsensor networks, we investigate the policy evaluation problem in a fullydecentralized setting, using temporal-difference (TD) learning with linearfunction approximation to handle large state spaces in practice. The goal of agroup of agents is to collaboratively learn the value function of a givenpolicy from locally private rewards observed in a shared environment, throughexchanging local estimates with neighbors. Despite their simplicity andwidespread use, our theoretical understanding of such decentralized TD learningalgorithms remains limited. Existing results were obtained based on i.i.d. datasamples, or by imposing an `additional' projection step to control the`gradient' bias incurred by the Markovian observations. In this paper, weprovide a finite-sample analysis of the fully decentralized TD(0) learningunder both i.i.d. as well as Markovian samples, and prove that all localestimates converge linearly to a small neighborhood of the optimum. Theresultant error bounds are the first of its type---in the sense that they holdunder the most practical assumptions ---which is made possible by means of anovel multi-step Lyapunov analysis.

Finite-Sample Analysis of Decentralized Temporal-Difference Learning with Linear Function Approximation