Two-Timescale Coordination of Discretely and Continuously Adjustable Devices in ADNs With DRL and Physical Convex Optimization

High penetration of electrical vehicles (EVs) and renewable distributed generators (DGs) into active distribution networks (ADNs) lead to frequent, rapid and fierce voltages magnitudes violations. A novel two-timescale coordination scheme for different types of adjustable devices in ADNs is put forward in this article by organically integrating data-driven deep reinforcement learning (DRL) into physical convex model. A Markov Decision Process (MDP) is formulated on slow timescale, in which ratios/statuses of on load transformer changers (OLTCs) and switchable capacitors reactors (SCRs) and ESSs charging/ discharging power are set hourly to optimize network losses while regulating voltages magnitudes. An improved DRL with relaxation-prediction-correction strategies is proposed for eradicating discrete action components dimension curses. Whereas, on fast timescale (e.g., several seconds or minutes), the optimal reactive power of DGs inverters and static VAR compensators (SVCs) in balanced and unbalanced ADNs are set with physical convex optimization to minimize network losses while respecting physical constraints. Five simulations cases with IEEE 33-node balanced and 123-node unbalanced feeders are carried out to verify capabilities of put forward method.