English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Value-based methods play a fundamental role in Markov decision processes(MDPs) and reinforcement learning (RL). In this paper, we present a unifiedcontrol-theoretic framework for analyzing valued-based methods such as valuecomputation (VC), value iteration (VI), and temporal difference (TD) learning(with linear function approximation). Built upon an intrinsic connectionbetween value-based methods and dynamic systems, we can directly use existingconvex testing conditions in control theory to derive various convergenceresults for the aforementioned value-based methods. These testing conditionsare convex programs in form of either linear programming (LP) or semidefiniteprogramming (SDP), and can be solved to construct Lyapunov functions in astraightforward manner. Our analysis reveals some intriguing connectionsbetween feedback control systems and RL algorithms. It is our hope that suchconnections can inspire more work at the intersection of system/control theoryand RL.

Convex Programs and Lyapunov Functions for Reinforcement Learning: A Unified Perspective on the Analysis of Value-Based Methods