Premium
Optimization of Real‐Time Reservoir Operations With Markov Decision Processes
Author(s) -
Wang Dapei,
Adams Barry J.
Publication year - 1986
Publication title -
water resources research
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.863
H-Index - 217
eISSN - 1944-7973
pISSN - 0043-1397
DOI - 10.1029/wr022i003p00345
Subject(s) - markov decision process , interim , mathematical optimization , markov process , computer science , markov chain , process (computing) , state (computer science) , steady state (chemistry) , markov model , mathematics , algorithm , statistics , machine learning , chemistry , archaeology , history , operating system
In recognition of hydrologic uncertainty and seasonality, reservoir inflows are described as periodic Markov processes. The optimization of reservoir operations involves determination of the optimal release volumes in the successive time periods so that the expected total rewards resulting from the operations are maximized. A two‐stage optimization framework, which consists of a real time model followed by a steady state model, is proposed. The steady state model that describes the convergent nature of the prospective future operations is regarded as a periodic Markov decision process and is optimized with the generalized policy iteration procedure. This result is in turn used as an interim step for deriving the optimal immediate decisions for the current period in the real‐time model. Significant computational efficiency results from this framework and the respective optimization procedure.