Using Reinforcement Learning to Control Life Support Systems

Advanced life support systems have many interacting processes and limited resources. Con- trolling and optimizing advanced life support systems presents unique challenges that are ad- dressed in this paper. In particular, advanced life support systems are nonlinear coupled dy- namical systems and it is dicult,for humans to take all interactions into account to design an eective control strategy. We have developed a controller using reinforcement learning [1], that actively explores the space of possible control strategies, guided by rewards from a user specified long term objective function. We evaluated this controller using a discrete event simulation of an advanced life support system. This simulation, called BioSim, has multiple, interacting life support modules including crew, food production, air revitalization, water recovery, solid waste incineration and power. These are implemented in a consumer/producer relationship in which certain modules produce resources that are consumed by other modules. Stores hold resources between modules. Control of this simulation is via adjusting flows of resources between modules and into/out of stores. This paper describes the results of using reinforcement learning to control the flow of resources in BioSim. Our technique discovered unobvious strategies for maximizing mission length. By exploiting non-linearities in the simulation dynamics, the learned controller outperforms a handwritten controller.