English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We discuss the approximation of the value function for infinite-horizondiscounted Markov Reward Processes (MRP) with nonlinear functions trained withthe Temporal-Difference (TD) learning algorithm. We first consider this problemunder a certain scaling of the approximating function, leading to a regimecalled lazy training. In this regime, the parameters of the model vary onlyslightly during the learning process, a feature that has recently been observedin the training of neural networks, where the scaling we study arisesnaturally, implicit in the initialization of their parameters. Both in theunder- and over-parametrized frameworks, we prove exponential convergence tolocal, respectively global minimizers of the above algorithm in the lazytraining regime. We then compare this scaling of the parameters to themean-field regime, where the approximately linear behavior of the model islost. Under this alternative scaling we prove that all fixed points of thedynamics in parameter space are global minimizers. We finally give examples ofour convergence results in the case of models that diverge if trained withnon-lazy TD learning, and in the case of neural networks.

Temporal-difference learning with nonlinear function approximation: lazy training and mean field regimes