Invisible Manipulation: Deep Reinforcement Learning-Enhanced Stealthy Attacks on Battery Energy Management Systems

This paper introduces an innovative cyber-attack scheme, "invisible manipulation," utilizing timed-stealthy false data injection attacks (Timed-SFDIAs). By subtly altering critical measurements ahead of a target period, the attacker covertly steers system operations toward a specific failure state, evading detection while enabling repeated attacks over time. Using Battery Energy Management System (BEMS) as a case study, we demonstrate the scheme's effectiveness in manipulating Battery Energy Storage Systems (BESS), critical for grids with high renewable penetration. Our method employs deep reinforcement learning (DRL) to generate synthetic measurements (e.g., battery voltage, current) that mimic real data, bypassing residual-based bad data detection (BDD) and misleading Extended Kalman-filter (EKF) based State-of-Charge (SoC) estimations. This allows the BEMS to operate the BESS per the attacker’s objectives. To minimize real-time computational demands, we transform this online optimization problem into an offline DRL training problem, utilizing high-fidelity simulation data from a digital twin-based microgrid testbed. The testbed incorporates real load and solar generation profiles with BESS models in the electromagnetic transient (EMT) domain at a 100-μs resolution, capturing rapid system dynamics and ensuring robust performance in real-time scenarios. Testing on the same testbed allows real-time evaluation of microgrid responses, where the BEMS, EKF-based SoC estimation algorithms interact dynamically with the injected false measurements. This unique DRL training and testing setup not only showcases the effectiveness of the Timed-SFDIA algorithm in evading detection and achieving diverse attack objectives but also underscores the critical role of high-fidelity, digital-twin based real-time simulation testbeds. Such testbeds are invaluable for training and validating data-driven machine learning algorithms, especially when field tests and real-world validation are challenging to conduct, as they ensure robustness and adaptability under realistic operational conditions.