English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Deep reinforcement learning (DRL) on Markov decision processes (MDPs) withcontinuous action spaces is often approached by directly training parametricpolicies along the direction of estimated policy gradients (PGs). Previousresearch revealed that the performance of these PG algorithms depends heavilyon the bias-variance tradeoffs involved in estimating and using PGs. A notableapproach towards balancing this tradeoff is to merge both on-policy andoff-policy gradient estimations. However existing PG merging methods can besample inefficient and are not suitable to train deterministic policiesdirectly. To address these issues, this paper introduces elite PGs andstrengthens their variance reduction effect by adopting elitism and policyconsolidation techniques to regularize policy training based on policybehavioral knowledge extracted from elite trajectories. Meanwhile, we propose atwo-step method to merge elite PGs and conventional PGs as a new extension ofthe conventional interpolation merging method. At both the theoretical andexperimental levels, we show that both two-step merging and interpolationmerging can induce varied bias-variance tradeoffs during policy training. Theyenable us to effectively use elite PGs and mitigate their performance impact ontrained policies. Our experiments also show that two-step merging canoutperform interpolation merging and several state-of-the-art algorithms on sixbenchmark control tasks.

Merging Deterministic Policy Gradient Estimations with Varied Bias-Variance Tradeoff for Effective Deep Reinforcement Learning