Multimodal Gesture Recognition Based on Attention Slow-Fast Fusion Networks

Gestures serve as the best alternative to traditional human-computer interaction (HCI), but there is still a great challenge to apply gestures to practical operations. Faced with the problem of generally low recognition accuracy in dynamic gesture recognition, we propose a fusion network with a slow-fast structure based on an attention mechanism to improve the recognition accuracy of dynamic gestures. The slow pathway acquires the temporal information of the input dynamic gesture, the fast pathway acquires the semantic information of the gesture in the input video, and suppresses the influence of non-gesture regions on the gesture features as much as possible through the attention mechanism, and finally performs the fusion operation according to the strategy of score fusion to obtain the recognition accuracy of the input dynamic gesture. We validate our proposed method on the ChaLearn large-scale gesture challenge gesture dataset IsoGD, and the experimental results are obtained to verify the effectiveness of our proposed structure by comparing it with the previous experimental results.