Hybrid face recognition under adverse conditions using appearance‐based and dynamic features of smile expression

Although recent deep‐learning‐based face recognition methods give remarkable accuracies on large databases, their performance has been shown to degrade under adverse conditions (e.g. severe illumination and contrast variations; blur and noise). Under such conditions, soft‐biometric features such as facial dynamics are expected to increase the performance if they are used together with appearance‐based features. We propose a novel hybrid face recognition, which uses appearance‐based features extracted using deep convolutional networks and statistical facial dynamics features extracted from facial landmark positions during smile expression. We evaluated the performances of three different state‐of‐the‐art pre‐trained deep convolutional neural networks (DCNNs) under a variety of severe image distortions with different parameters. The experimental results show that, although the face recognition performance using only DCNN‐based features drops significantly under adverse conditions, the utilization of facial dynamics features together with DCNN‐based features can compensate for the performance loss and increase the accuracy significantly. We believe the proposed system can be useful when face recognition is performed using videos obtained from systems, which may contain blurry and noisy images with a wide range of illumination variations.