English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many complex domains, such as robotics control and real-time strategy (RTS)games, require an agent to learn a continuous control. In the former, an agentlearns a policy over $\mathbb{R}^d$ and in the latter, over a discrete set ofactions each of which is parametrized by a continuous parameter. Such problemsare naturally solved using policy based reinforcement learning (RL) methods,but unfortunately these often suffer from high variance leading to instabilityand slow convergence. Unnecessary variance is introduced whenever policies overbounded action spaces are modeled using distributions with unbounded support byapplying a transformation $T$ to the sampled action before execution in theenvironment. Recently, the variance reduced clipped action policy gradient(CAPG) was introduced for actions in bounded intervals, but to date no variancereduced methods exist when the action is a direction, something often seen inRTS games. To this end we introduce the angular policy gradient (APG), astochastic policy gradient method for directional control. With the marginalpolicy gradients family of estimators we present a unified analysis of thevariance reduction properties of APG and CAPG; our results provide a strongerguarantee than existing analyses for CAPG. Experimental results on a popularRTS game and a navigation task show that the APG estimator offers a substantialimprovement over the standard policy gradient.

Marginal Policy Gradients: A Unified Family of Estimators for Bounded Action Spaces with Applications