English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

We consider infinite-horizon $\gamma$-discounted Markov Decision Processes,for which it is known that there exists a stationary optimal policy. Weconsider the algorithm Value Iteration and the sequence of policies$\pi_1,...,\pi_k$ it implicitely generates until some iteration $k$. We provideperformance bounds for non-stationary policies involving the last $m$ generatedpolicies that reduce the state-of-the-art bound for the last stationary policy$\pi_k$ by a factor $\frac{1-\gamma}{1-\gamma^m}$. In particular, the use ofnon-stationary policies allows to reduce the usual asymptotic performancebounds of Value Iteration with errors bounded by $\epsilon$ at each iterationfrom $\frac{\gamma}{(1-\gamma)^2}\epsilon$ to$\frac{\gamma}{1-\gamma}\epsilon$, which is significant in the usual situationwhen $\gamma$ is close to 1. Given Bellman operators that can only be computedwith some error $\epsilon$, a surprising consequence of this result is that theproblem of "computing an approximately optimal non-stationary policy" is muchsimpler than that of "computing an approximately optimal stationary policy",and even slightly simpler than that of "approximately computing the value ofsome fixed policy", since this last problem only has a guarantee of$\frac{1}{1-\gamma}\epsilon$.

On the Use of Non-Stationary Policies for Infinite-Horizon Discounted Markov Decision Processes