English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Trust region policy optimization (TRPO) is a popular and empiricallysuccessful policy search algorithm in Reinforcement Learning (RL) in which asurrogate problem, that restricts consecutive policies to be 'close' to oneanother, is iteratively solved. Nevertheless, TRPO has been considered aheuristic algorithm inspired by Conservative Policy Iteration (CPI). We showthat the adaptive scaling mechanism used in TRPO is in fact the natural "RLversion" of traditional trust-region methods from convex analysis. We firstanalyze TRPO in the planning setting, in which we have access to the model andthe entire state space. Then, we consider sample-based TRPO and establish$\tilde O(1/\sqrt{N})$ convergence rate to the global optimum. Importantly, theadaptive scaling mechanism allows us to analyze TRPO in regularized MDPs forwhich we prove fast rates of $\tilde O(1/N)$, much like results in convexoptimization. This is the first result in RL of better rates when regularizingthe instantaneous cost or reward.

Adaptive Trust Region Policy Optimization: Global Convergence and Faster Rates for Regularized MDPs