Deep multi-factor forensic face recognition

of the Dissertation Deep Multi-Factor Forensic Face Recognition by Zhengming Ding Doctor of Philosophy in Electrical and Computer Engineering Northeastern University, 2018 Dr. Yun Fu, Advisor Forensic science is any scientific field that is applied to the field of law. Due to the popularity of the digital media carriers such as images, videos, the facial recognition becomes another important forensic technique. The major issue of forensic face recognition is the unstable system performance due to internal factor, e.g., aging, and external factors, e.g., image resolution/modality, illumination, pose. In this thesis, we investigate a theoretical framework for forensic face recognition, subject to a variety of internal/external impact factors to tackle face recognition under different views, illuminations, resolutions, modalities, periods, when probe images are captured in the surveillance environments without collaborations. Specifically, we explore two scenarios as follows. First of all, we explore one dominant factor which hinders the forensic face recognition, which is view variance, e.g., pose and modality. Thus, we propose multi-view face recognition, which covers two settings in multi-view face recognition. On one hand, labeled data in multiple views are available in the training stage, which is the traditional multi-view learning setting. Specifically, we address the challenging but practical situation, in which the view information of the test data is unknown. On the other hand, some source views are labeled while target views are unlabeled, which satisfies transfer learning scenarios. Specifically, we explore the practical but challenging missing modality problem. Secondly, multiple factors are modeled as the noises as a whole. On one hand, conventional auto-encoder and its variants usually involve additive noises for training data to learn robust features, which, however, did not consider the already corrupted data. We propose a novel Deep Robust Encoder (DRE) through locality preserving low-rank dictionary to extract robust and discriminative features from corrupted data. Furthermore, we fight off one-shot face recognition, where we only have one training sample for some persons, by proposing a one-shot generative model to build a more effective face recognizer.