Optimal Cogeneration Scheduling: A Comparison of Genetic and POMDP-based Deep Reinforcement Learning Approaches

Large processing facilities require multiple types of energy, such as electrical and thermal (hot water or steam). Cogeneration, or Combined Heat and Power (CHP), can provide significant economic and energy savings. However, scheduling its operation in real-time is challenging. This work compares deep reinforcement learning (DRL) and genetic algorithm (GA) approaches to control a real CHP in a processing facility. Traditionally, the CHP economic dispatch problem is modelled as a Markov Decision Process (MDP) with the assumption of complete observability. Due to the uncertainty of future electric and thermal demands, this assumption is unrealistic in real-world scenarios. Thus, this work proposes using a partially observable MDP (POMDP) for hourly CHP dispatch scheduling to address this partial observability. The selected DRL algorithms are Deep Q Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Soft Actor-Critic (SAC), along with six GA variants. Performance was evaluated using multiple economic metrics, including Earnings Before Interest, Taxes, Depreciation, and Amortization (EBITDA), an environmental analysis, and a sensitivity analysis under variable electric pricing. This work shows that POMDP effectively models the hourly dispatch scheduling problem of CHPs. The insights gained from this analysis offer multiple potential avenues for future research, including the development more advanced DRL algorithms for CHP economic dispatch and the evaluation of their resilience when inaccurate measurements and anomalous conditions occur.