Real-time Energy Management Based on Proximal Policy Optimization with Mask Layer for Hybrid Electric Mining Trucks

An effective energy management strategy (EMS) is crucial to improve the energy efficiency of hybrid vehicles, especially for heavy-duty mining trucks. An energy management strategy based on a proximal policy optimization algorithm with mask layer and novel reward functions (PPO-MASK-NR) is proposed for hybrid electric mining trucks (HEMTs) with multi-planetary systems. This algorithm fundamentally avoids irrational exploration by an intelligent agent by incorporating a real-time mask layer, and it accelerates learning efficiency by suppressing the backward propagation of gradients for irrational actions. A universally designed reward function is applied to ensure the achievement of the correct final state of charge (SOC) value and the expansion of the SOC's exploration range. Finally, the generalization performance of the proposed algorithm is validated through new driving cycles, and its authenticity is confirmed through hardware-in-the-loop (HiL) testing. The simulation results show that within the selected training cycles, the proposed algorithm achieves 98% compared with the dynamic programming algorithm (DP). The proposed algorithm has an improvement of 11% and 5% in online applications for a new driving cycle compared to a rule-based technique (RB) and the equivalent fuel consumption minimization approach (ECMS), respectively.