English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Consider the problem of approximating the optimal policy of a Markov decisionprocess (MDP) by sampling state transitions. In contrast to existingreinforcement learning methods that are based on successive approximations tothe nonlinear Bellman equation, we propose a Primal-Dual $\pi$ Learning methodin light of the linear duality between the value and policy. The $\pi$ learningmethod is model-free and makes primal-dual updates to the policy and valuevectors as new data are revealed. For infinite-horizon undiscounted Markovdecision process with finite state space $S$ and finite action space $A$, the$\pi$ learning method finds an $\epsilon$-optimal policy using the followingnumber of sample transitions $$ \tilde{O}( \frac{(\tau\cdot t^*_{mix})^2 |S||A| }{\epsilon^2} ),$$ where $t^*_{mix}$ is an upper bound of mixing timesacross all policies and $\tau$ is a parameter characterizing the range ofstationary distributions across policies. The $\pi$ learning method alsoapplies to the computational problem of MDP where the transition probabilitiesand rewards are explicitly given as the input. In the case where each statetransition can be sampled in $\tilde{O}(1)$ time, the $\pi$ learning methodgives a sublinear-time algorithm for solving the averaged-reward MDP.

Primal-Dual $\pi$ Learning: Sample Complexity and Sublinear Run Time for Ergodic Markov Decision Problems