Open AccessHierarchical CNN‐based real‐time fatigue detection system by visual‐based technologies using MSP model
Author(s) -
Gu Wang Huan,
Zhu Yu,
Chen Xu Dong,
He Lin Fei,
Zheng Bing Bing
Publication year - 2018
Publication title -
iet image processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.401
H-Index - 45
eISSN - 1751-9667
pISSN - 1751-9659
DOI - 10.1049/iet-ipr.2018.5245
Subject(s) - computer science , convolutional neural network , artificial intelligence , computer vision , face detection , task (project management) , pooling , intersection (aeronautics) , visualization , pattern recognition (psychology) , facial recognition system , engineering , systems engineering , aerospace engineering
Visual‐based technologies are very useful and meaningful to driver's fatigue detection. In this study, the authors present a multi‐task hierarchical CNN scheme for fatigue detection system and propose a convolutional neural network (CNN) model with multi‐scale pooling (MSP‐Net). ‘Multi‐task’ includes three tasks: face detection, eye and mouth state detection and fatigue detection. First, they use a pre‐trained network – multi‐task CNN for face detection extracting eye and mouth regions. Then, the main work of this study, eye and mouth state detection is processed by MSP‐Net, which can fit multi‐resolution input images captured from variant cameras excellently. For the third step, the percentage of eyelid closure over the pupil over time (PERCLOS) parameters and the frequency of open mouth (FOM) parameters are used to detect fatigue, and the FOM parameters are proposed by ourselves. Besides, they successfully port the system to the embedded platform (the NVIDIA JETSON TX2 development board) and test on real driving scene. The results show that their system performs well and is robust to complex environments and is in line with the demand of real‐time system.