Pain intensity recognition via multi‐scale deep network

Similar to the basic facial expression recognition, one challenge for pain intensity recognition is some individual characteristics, e.g. face shapes, may cause great diversities in the same emotion. So it is usually very difficult to distinguish two adjacent intensity levels of pain expression as each intensity has a large variation. In this study, a coarse‐to‐fine combination method is proposed for pain intensity recognition. The results of multi‐scale outputs from multiple base deep network are combined in a probabilistic way for improving the discrimination between visually similar adjacent levels. A two‐layer tree classifier is proposed in a multi‐task framework for pain intensity recognition as well as face shape recognition, replacing the planar Softmax classifier in each base deep network. In the first layer of tree classifier, multi‐scale classifiers are constructed for recognizing facial pain intensities and the conventional classifiers are constructed for face shape recognition in the second layer. Finally, the tree classifier including multi‐scale classifiers and conventional classifiers is jointly optimised during the training phase and only high level classifiers are used for recognising pain intensities in the test phase. The extensive experiments on UNBC shoulder pain dataset show the proposed method gets promising results in pain intensity recognition.