Open AccessEEG-based emotion recognition using 4D convolutional recurrent neural network
Author(s) -
Fangyao Shen,
Guojun Dai,
Guang Lin,
Jianhai Zhang,
Wanzeng Kong,
Hong Zeng
Publication year - 2020
Publication title -
cognitive neurodynamics
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.83
H-Index - 41
eISSN - 1871-4099
pISSN - 1871-4080
DOI - 10.1007/s11571-020-09634-1
Subject(s) - computer science , convolutional neural network , electroencephalography , artificial intelligence , recurrent neural network , pattern recognition (psychology) , emotion recognition , deep learning , speech recognition , artificial neural network , psychology , psychiatry
In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition.