Open AccessArousal‐valence recognition using CNN with STFT feature‐combined image
Author(s) -
Lee H.J.,
Lee S.G.
Publication year - 2018
Publication title -
electronics letters
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.375
H-Index - 146
ISSN - 1350-911X
DOI - 10.1049/el.2017.3538
Subject(s) - artificial intelligence , convolutional neural network , pattern recognition (psychology) , feature (linguistics) , computer science , short time fourier transform , feature extraction , computer vision , fourier transform , speech recognition , mathematics , fourier analysis , mathematical analysis , philosophy , linguistics
A novel ocular‐features‐combining method, called short‐time Fourier transform (STFT) feature‐combined image, and a simple convolutional neural networks (CNNs) model are proposed for arousal‐valence recognition. The STFT feature‐combined image aims to represent information on two ocular features (pupil size and eye movements) as a single image. The CNN model consists of two convolutional layers and uses STFT feature‐combined image as an input. The experimental results demonstrate the effectiveness of the proposed method, and show that CNN model is not only effective for emotion‐recognition methods based on other modalities, but also effective for ocular‐feature‐based emotion recognition.