Real-time Snowfall Recognition and Object Tracking System for Autonomous Driving in Adverse Snowy Weather

Human can drive safely by instinctively accounting for real-time environmental and weather conditions such as snow and rain. In contrast, current autonomous driving systems still face challenges in achieving full autonomy, as their path planning and control rely solely on environmental perception without considering weather conditions. However, to realize fully autonomous driving, it is essential to implement vehicle control technologies that can recognize not only the surrounding environment but also weather conditions, and use this information to adjust steering and longitudinal behaviors—such as acceleration and deceleration—in a human-like manner. To address this issue, this study proposes a perception system capable of recognizing both environmental and weather conditions in real time under snowy weather, enabling weather-aware vehicle control. This is the first perception system to simultaneously provide both object and weather information. It improves detection performance for snow-affected cameras through YOLOv4 and YOLOv8-based training, and enhances LiDAR accuracy by removing snow-induced noise. Robust radar data, resistant to environmental changes, are then fused using a Kalman filter to enable reliable object tracking. Finally, weather information is inferred from the number of real-time filtered LiDAR points, allowing the system to perform simultaneous object tracking and environmental perception in snowy conditions. Compared with existing tracking systems that are not specialized for extreme environments, the proposed system offers more than 50% improved tracking accuracy, accurate snowfall classification with a precision of 0.983, and more than 15% faster processing time. This enables precise path planning and allows for safe vehicle control during steering, acceleration and deceleration by utilizing real-time weather information. This study provides valuable real-time data that enable the seamless operation of all essential components of autonomous driving technology, such as sensors, perception, path planning, and vehicle control, even under snowy conditions, therefore enhancing the potential to achieve Level 5 autonomy.