Premium
Measuring facial expressions by computer image analysis
Author(s) -
Bartlett Marian Stewart,
Hager Joseph C.,
Ekman Paul,
Sejnowski Terrence J.
Publication year - 1999
Publication title -
psychophysiology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 1.661
H-Index - 156
eISSN - 1469-8986
pISSN - 0048-5772
DOI - 10.1017/s0048577299971664
Subject(s) - facial action coding system , facial expression , artificial intelligence , psychology , task (project management) , cognition , pattern recognition (psychology) , computer vision , computer science , neuroscience , management , economics
Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System (Ekman & Friesen, 1978) is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.