English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider the estimation of the policy gradient in partially observableMarkov decision processes (POMDP) with a special class of structured policiesthat are finite-state controllers. We show that the gradient estimation can bedone in the Actor-Critic framework, by making the critic compute a "value"function that does not depend on the states of POMDP. This function is theconditional mean of the true value function that depends on the states. We showthat the critic can be implemented using temporal difference (TD) methods withlinear function approximations, and the analytical results on TD andActor-Critic can be transfered to this case. Although Actor-Critic algorithmshave been used extensively in Markov decision processes (MDP), up to now theyhave not been proposed for POMDP as an alternative to the earlier proposalGPOMDP algorithm, an actor-only method. Furthermore, we show that the same ideaapplies to semi-Markov problems with a subset of finite-state controllers.

A Function Approximation Approach to Estimation of Policy Gradient for POMDP with Structured Policies