English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

We study policy gradient (PG) for reinforcement learning in continuous timeand space under the regularized exploratory formulation developed by Wang etal. (2020). We represent the gradient of the value function with respect to agiven parameterized stochastic policy as the expected integration of anauxiliary running reward function that can be evaluated using samples and thecurrent value function. This effectively turns PG into a policy evaluation (PE)problem, enabling us to apply the martingale approach recently developed by Jiaand Zhou (2021) for PE to solve our PG problem. Based on this analysis, wepropose two types of the actor-critic algorithms for RL, where we learn andupdate value functions and policies simultaneously and alternatingly. The firsttype is based directly on the aforementioned representation which involvesfuture trajectories and hence is offline. The second type, designed for onlinelearning, employs the first-order condition of the policy gradient and turns itinto martingale orthogonality conditions. These conditions are thenincorporated using stochastic approximation when updating policies. Finally, wedemonstrate the algorithms by simulations in two concrete examples.

Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms