English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We study a reinforcement learning setting, where the state transitionfunction is a convex combination of a stochastic continuous function and adeterministic function. Such a setting generalizes the widely-studiedstochastic state transition setting, namely the setting of deterministic policygradient (DPG).  We firstly give a simple example to illustrate that the deterministic policygradient may be infinite under deterministic state transitions, and introduce atheoretical technique to prove the existence of the policy gradient in thisgeneralized setting. Using this technique, we prove that the deterministicpolicy gradient indeed exists for a certain set of discount factors, andfurther prove two conditions that guarantee the existence for all discountfactors. We then derive a closed form of the policy gradient whenever exists.Furthermore, to overcome the challenge of high sample complexity of DPG in thissetting, we propose the Generalized Deterministic Policy Gradient (GDPG)algorithm. The main innovation of the algorithm is a new method of applyingmodel-based techniques to the model-free algorithm, the deep deterministicpolicy gradient algorithm (DDPG). GDPG optimize the long-term rewards of themodel-based augmented MDP subject to a constraint that the long-rewards of theMDP is less than the original one.  We finally conduct extensive experiments comparing GDPG with state-of-the-artmethods and the direct model-based extension method of DDPG on several standardcontinuous control benchmarks. Results demonstrate that GDPG substantiallyoutperforms DDPG, the model-based extension of DDPG and other baselines interms of both convergence and long-term rewards in most environments.

Deterministic Policy Gradients With General State Transitions