English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Proximal policy optimization and trust region policy optimization (PPO andTRPO) with actor and critic parametrized by neural networks achieve significantempirical success in deep reinforcement learning. However, due to nonconvexity,the global convergence of PPO and TRPO remains less understood, which separatestheory from practice. In this paper, we prove that a variant of PPO and TRPOequipped with overparametrized neural networks converges to the globallyoptimal policy at a sublinear rate. The key to our analysis is the globalconvergence of infinite-dimensional mirror descent under a notion of one-pointmonotonicity, where the gradient and iterate are instantiated by neuralnetworks. In particular, the desirable representation power and optimizationgeometry induced by the overparametrization of such neural networks allow themto accurately approximate the infinite-dimensional gradient and iterate.

Neural Proximal/Trust Region Policy Optimization Attains Globally Optimal Policy