English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Despite the empirical success of the actor-critic algorithm, its theoreticalunderstanding lags behind. In a broader context, actor-critic can be viewed asan online alternating update algorithm for bilevel optimization, whoseconvergence is known to be fragile. To understand the instability ofactor-critic, we focus on its application to linear quadratic regulators, asimple yet fundamental setting of reinforcement learning. We establish anonasymptotic convergence analysis of actor-critic in this setting. Inparticular, we prove that actor-critic finds a globally optimal pair of actor(policy) and critic (action-value function) at a linear rate of convergence.Our analysis may serve as a preliminary step towards a complete theoreticalunderstanding of bilevel optimization with nonconvex subproblems, which isNP-hard in the worst case and is often solved using heuristics.

On the Global Convergence of Actor-Critic: A Case for Linear Quadratic Regulator with Ergodic Cost