Tracking Control of CSTRs Based on Improved OU Noise and the TD3 Algorithm

Achieving satisfactory control accuracy in typical chemical processes is a challenging task. This is primarily due to their complex nonlinear dynamic characteristics. To address this challenge, this study proposes a novel deep reinforcement learning (DRL) approach that integrates an improved Ornstein-Uhlenbeck (IOU) noise into the twin delayed deep deterministic policy gradient (TD3) algorithm. This method is applied to the tracking control of continuous stirred tank reactors (CSTRs). Initially, a mechanistic model of the CSTRs system is established to simulate its dynamic environment, enabling interaction between the TD3 agent and the system. To enhance exploration capabilities and convergence speed, fractional-order characteristics and a reward feedback mechanism are introduced into the OU noise, dynamically adjusting noise intensity to improve adaptability to complex states and optimize the exploration strategy of the TD3 algorithm. Furthermore, a well-designed reward function and optimized hyperparameters enable the agent to efficiently learn the optimal control policy, achieving high-precision tracking control of the CSTRs system. Simulation results demonstrate that the proposed TD3 algorithm outperforms conventional control methods, such as PID control and nonlinear model predictive control, as well as other DRL algorithms, including SAC and PPO, in terms of control accuracy and convergence speed.