Open AccessFusing HOG and convolutional neural network spatial–temporal features for video‐based facial expression recognition
Author(s) -
Pan Xianzhang
Publication year - 2020
Publication title -
iet image processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.401
H-Index - 45
eISSN - 1751-9667
pISSN - 1751-9659
DOI - 10.1049/iet-ipr.2019.0293
Subject(s) - convolutional neural network , artificial intelligence , computer science , pattern recognition (psychology) , facial expression , histogram of oriented gradients , support vector machine , feature (linguistics) , histogram , computer vision , three dimensional face recognition , facial expression recognition , feature extraction , facial recognition system , image (mathematics) , face detection , linguistics , philosophy
Video‐based facial expression recognition (VFER) is the fundamental feature of various computer vision applications. Visual features are the key factors for facial expression recognition. However, the gap between the visual features and the emotions is large. In order to bridge the gap, the proposed method utilises convolutional neural networks (CNNs) and histogram of oriented gradient (HOG) to obtain the more comprehensive feature for VFER. Firstly, it extracts shallow features from the video frame through a number of convolutional kernels in CNNs, which has the characteristics of displacement, scale and deformation invariance. Then, the HOG is employed to extract HOG features from CNN's shallow features, which are strongly correlated with facial expressions. Finally, the support vector machine (SVM) is employed to conduct the task of facial expression recognition. The extensive experiments on RML, CK+ and AFEW5.0 database show that this framework takes on the promising performance and outperforming the state of the arts.