Open AccessA Moreau-Yosida regularization for Markov decision processes
Author(s) -
R. Israel Ortega-Gutiérrez,
Hugo Cruz−Suárez
Publication year - 2020
Publication title -
proyecciones
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.226
H-Index - 12
eISSN - 0717-6279
pISSN - 0716-0917
DOI - 10.22199/issn.0717-6279-2021-01-0008
Subject(s) - markov decision process , markov process , mathematical optimization , bellman equation , mathematics , markov kernel , partially observable markov decision process , markov chain , uniqueness , markov property , regularization (linguistics) , markov model , computer science , mathematical economics , variable order markov model , artificial intelligence , statistics , mathematical analysis
This paper addresses a class of sequential optimization problems known as Markov decision processes. These kinds of processes are considered on Euclidean state and action spaces with the total expected discounted cost as the objective function. The main goal of the paper is to provide conditions to guarantee an adequate Moreau-Yosida regularization for Markov decision processes (named the original process). In this way, a new Markov decision process that conforms to the Markov control model of the original process except for the cost function induced via the Moreau-Yosida regularization is established. Compared to the original process, this new discounted Markov decision process has richer properties, such as the differentiability of its optimal value function, strictly convexity of the value function, uniqueness of optimal policy, and the optimal value function and the optimal policy of both processes, are the same. To complement the theory presented, an example is provided.