English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Markov Decision Processes (MDP) is an useful framework to cast optimalsequential decision making problems. Given any MDP the aim is to find theoptimal action selection mechanism i.e., the optimal policy. Typically, theoptimal policy ($u^*$) is obtained by substituting the optimal value-function($J^*$) in the Bellman equation. Alternately $u^*$ is also obtained by learningthe optimal state-action value function $Q^*$ known as the $Q$ value-function.However, it is difficult to compute the exact values of $J^*$ or $Q^*$ for MDPswith large number of states. Approximate Dynamic Programming (ADP) methodsaddress this difficulty by computing lower dimensional approximations of$J^*$/$Q^*$. Most ADP methods employ linear function approximation (LFA), i.e.,the approximate solution lies in a subspace spanned by a family of pre-selectedbasis functions. The approximation is obtain via a linear least squaresprojection of higher dimensional quantities and the $L_2$ norm plays animportant role in convergence and error analysis. In this paper, we discuss ADPmethods for MDPs based on LFAs in $(\min,+)$ algebra. Here the approximatesolution is a $(\min,+)$ linear combination of a set of basis functions whosespan constitutes a subsemimodule. Approximation is obtained via a projectionoperator onto the subsemimodule which is different from linear least squaresprojection used in ADP methods based on conventional LFAs. MDPs are not$(\min,+)$ linear systems, nevertheless, we show that the monotonicity propertyof the projection operator helps us to establish the convergence of our ADPschemes. We also discuss future directions in ADP methods for MDPs based on the$(\min,+)$ LFAs.

Approximate dynamic programming with $(\min,+)$ linear function approximation for Markov decision processes