A Methodology for Automated Facial Expression Recognition Using Facial Landmarks

Facial expression recognition is a crucial part of Psychology as a person's facial expression accounts for 55 percent of the effect of a spoken message. This makes facial expression the single biggest indicator of individual communication. Traditionally, Psychologists trained human observers to identify changes in facial muscles and use a Facial Action Coding System to map muscle movements to an emotion. Though this system helped ensure objectivity and had descriptive power, its major drawback was in effectively training human observers. With the advent of faster computers and the use of pixels/megapixels for picture elements, machine learning researchers became interested in automating facial expression recognition. Most researchers continued adopting the same Facial Action Coding System, used in Psychology, to train their statistical models. Though advances have been made in automating facial detection and finding facial landmarks, facial expression recognition results have stagnated. This stagnation is blamed on lack of training data and the difficulty of training a model to recognize subtle changes in facial muscles. In this paper the authors describe how software engineering best practices assisted in developing and implementing a methodology to leverage larger facial detection and facial landmarking datasets, as well as their improved accuracy over the Facial Action Coding System. This methodology is potentially more descriptive of faces in unconstrained environments. The authors present the software artifacts, the methodology, and the findings of a comparative study.