English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Natural policy gradient methods are popular reinforcement learning methodsthat improve the stability of policy gradient methods by utilizing second-orderapproximations to precondition the gradient with the inverse of theFisher-information matrix. However, to the best of the authors' knowledge,there has not been a study that has investigated the effects of differentsecond-order approximations in a comprehensive and systematic manner. Toaddress this, five different second-order approximations were studied andcompared across multiple key metrics including performance, stability, sampleefficiency, and computation time. Furthermore, hyperparameters which aren'ttypically acknowledged in the literature are studied including the effect ofdifferent batch sizes and optimizing the critic network with the naturalgradient. Experimental results show that on average, improved second-orderapproximations achieve the best performance and that using properly tunedhyperparameters can lead to large improvements in performance and sampleefficiency ranging up to +181%. We also make the code in this study availableat https://github.com/gebob19/natural-policy-gradient-reinforcement-learning.

Understanding the Effects of Second-Order Approximations in Natural Policy Gradient Reinforcement Learning