English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent analyses of certain gradient descent optimization methods have shownthat performance can degrade in some settings - such as with stochasticity orimplicit momentum. In deep reinforcement learning (Deep RL), such optimizationmethods are often used for training neural networks via the temporal differenceerror or policy gradient. As an agent improves over time, the optimizationtarget changes and thus the loss landscape (and local optima) change. Due tothe failure modes of those methods, the ideal choice of optimizer for Deep RLremains unclear. As such, we provide an empirical analysis of the effects thata wide range of gradient descent optimizers and their hyperparameters have onpolicy gradient methods, a subset of Deep RL algorithms, for benchmarkcontinuous control tasks. We find that adaptive optimizers have a narrow windowof effective learning rates, diverging in other cases, and that theeffectiveness of momentum varies depending on the properties of theenvironment. Our analysis suggests that there is significant interplay betweenthe dynamics of the environment and Deep RL algorithm properties which aren'tnecessarily accounted for by traditional adaptive gradient methods. We providesuggestions for optimal settings of current methods and further lines ofresearch based on our findings.

Where Did My Optimum Go?: An Empirical Analysis of Gradient Descent Optimization in Policy Gradient Methods