English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Many problems faced by decision makers today involve the management of large scale, complex systems that can be modeled as state-based control problems, specifically discrete Markov decision process (MDP). Typical examples include transportation systems, defense systems, healthcare networks, financial organizations, and general infrastructure problems. In all of these problems, decision makers have difficulty in forecasting the state of their system in the future and capturing the dynamics of the states over time. In this paper, we discuss, via numerous examples, practical experiences in trying to build such models. Much of the literature discusses theoretical issues of solution convergence and algorithm performance; unfortunately, much of this research does not help with the practical business of building an actual MDP model. Thus, numerous books begin with statement of the nature: "given the state space S. . .." A critical question to the practitioner is the creation of this state space "S." We focus on this first step in the MDP modeling process, an often neglected and difficult step, and we discuss the practical implications and issues associated with the state definition, illustrating these issues with numerous examples. This paper is not meant to be a survey of "state-based" applications or MDP applications, but an overview of experiences building many of these models in diverse applications.

On the Practical Art of State Definitions for Markov Decision Process Construction