Adaptive Cruise Control Based on Reinforcement Leaning with Shaping Rewards

This paper proposes a Supervised Reinforcement Learning (SRL) algorithm for the Adaptive Cruise Control system (ACC) to comply with human driving habit, which can be thought of as a dynamic programming problem with stochastic demands. In short, the ACC problem can be deemed as the host vehicle adopting different control parameters (accelerations in the upper controller, brakes and throttles in the bottom controller) in the process of following or other driving situations according to the driver’s behavior. We discretize the relative speed as well as the relative distance to construct the two-dimensional states and map them to a one-dimensional state space; discretize the acceleration to generate the action set; and design additional speed improvement shaping reward and distance improvement shaping reward to construct the supervisor. We apply the SRL algorithm to the ACC problem in different scenarios. The results show the higher robustness and accuracy of the SRL control policy compared with traditional control methods.