Open AccessWearable on-device deep learning system for hand gesture recognition based on FPGA accelerator
Author(s) -
Wenming Jiang,
Xuelin Ye,
Ruiqi Chen,
Feng Su,
Mengru Lin,
Yuhanxiao Ma,
Yanxiang Zhu,
Shizhen Huang
Publication year - 2021
Publication title -
mathematical biosciences and engineering
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.451
H-Index - 45
eISSN - 1551-0018
pISSN - 1547-1063
DOI - 10.3934/mbe.2021007
Subject(s) - computer science , gesture recognition , inertial measurement unit , field programmable gate array , wearable computer , gesture , convolutional neural network , artificial intelligence , deep learning , microcontroller , multilayer perceptron , artificial neural network , computer hardware , embedded system
Gesture recognition is critical in the field of Human-Computer Interaction, especially in healthcare, rehabilitation, sign language translation, etc. Conventionally, the gesture recognition data collected by the inertial measurement unit (IMU) sensors is relayed to the cloud or a remote device with higher computing power to train models. However, it is not convenient for remote follow-up treatment of movement rehabilitation training. In this paper, based on a field-programmable gate array (FPGA) accelerator and the Cortex-M0 IP core, we propose a wearable deep learning system that is capable of locally processing data on the end device. With a pre-stage processing module and serial-parallel hybrid method, the device is of low-power and low-latency at the micro control unit (MCU) level, however, it meets or exceeds the performance of single board computers (SBC). For example, its performance is more than twice as much of Cortex-A53 (which is usually used in Raspberry Pi). Moreover, a convolutional neural network (CNN) and a multilayer perceptron neural network (NN) is used in the recognition model to extract features and classify gestures, which helps achieve a high recognition accuracy at 97%. Finally, this paper offers a software-hardware co-design method that is worth referencing for the design of edge devices in other scenarios.