Efficient Detection of Targeted Adversarial Attacks in Automatic Speech Recognition Systems

Automatic Speech Recognition (ASR) systems convert spoken words into text and play a critical role in voice assistants, transcription services, and other accessibility tools. However, their reliance on machine learning models makes them susceptible to adversarial attacks. By adding small, often imperceptible perturbations to the input audio, or by convolving it with a carefully designed room impulse response, attackers can cause ASR systems to produce incorrect or even malicious transcripts. In this work, we propose an ensemble detection method for the efficient identification of targeted adversarial attacks in ASR systems. Specifically, we introduce MELo-FEST, which calculates the minimum energy in the low-frequency band of an audio signal to detect convolutive adversarial attacks. We then combine two spectrogram-based detection methods with a noise-adding approach to form an ensemble detector capable of identifying both additive and convolutive targeted adversarial attacks. Through extensive experiments, we demonstrate that our proposed ensemble detector can accurately identify adversarial audio generated by both non-adaptive and adaptive CW, PGD, and AdvReverb attacks in Wav2Vec2 and Whisper ASR systems, achieving an F1 score of at least 0.97. Moreover, our method performs both speech recognition and adversarial detection for each input audio sample in an average of under 0.13 seconds for Wav2Vec2 and 0.29 seconds for Whisper, making it well-suited for real-time applications. Additionally, we find that Whisper is more vulnerable than Wav2Vec2 to both non-adaptive and adaptive targeted adversarial attacks.