Modified deep belief network based human emotion recognition with multiscale features from video sequences

Summary Emotion recognition from human faces are recently considered as growing topic for the applications in HCI (human–computer interaction) field. Therefore, a new framework is introduced in this method for emotion recognition from video. Human faces may carry huge features which increase the complexity of recognizing the emotions from the give video. Therefore, to minimize such defect, the wrapper based feature selection technique is introduced which reduce the complexity of proposed recognition framework. Initially, the frames from the input video is preprocessed. Next, the features exhibited by each emotions are extracted with geometric and local binary pattern‐based feature extraction methods. Then, the features that reduce the performance of recognition technique is avoided using a feature selection algorithm. It selects the features that provides effective result on recognition process. Finally, the selected features are provided to deep belief network (DBN) for emotion recognition. The weight parameter selection of DBN is improved using an efficient Harris Hawk optimization algorithm. The performance of presented architecture is evaluated using a three different datasets they are FAMED, CK+, and MMI. The overall rate shown by proposed architecture is found better than existing methods. Furthermore, the precision, recall, and specificity are also evaluated for six different emotions (angry, disgust, fear, happy, sad, and surprise) in this proposed method. This entire emotion recognition process is implemented in Python platform.