FPGA Platform applied for Facial Expression Recognition System using Convolutional Neural Networks

Emotion being a subjective thing, leveraging knowledge and science behind labeled data and extracting the components that constitute it. With the development of deep learning in computer vision, emotion recognition has become a widely-tackled research problem. In this work, we propose a Field Programmable Gate Array (FPGA) architecture applied for this task using independent method called convolutional neural network (CNN). The emotion recognition block receives the detected faces from a video stream by using VITA-2000 camera module and process the image data with the trained CNN model. The architecture is implemented on a Zynq-7000 All Programmable SoC Video and Imaging Kit. Once we have trained a network, weights from the Tensorflow model will be convert as C-arrays, to be used in Vivado HLS. After having the weights as C arrays, they can be implemented to FPGA system. We can also test the functionality of the CNN entirely, by compiling the design with C++ compiler. This method was trained on the posed-emotion dataset (FER2013). The results show that with more fine-tuning and depth, the CNN model can outperform the state-of-the-art methods for emotion recognition. We also propose some exciting ideas for expanding the concept of representational landmark features and sliding windows to improve its performance.