Face Alignment Through 2.5D Active Appearance Models

This work addresses the fitting of 3D deformable face models from a single view through 2.5D Active Appearance Models (AAM). The main contribution of this paper is the use of 2.5D AAM that combines a 3D metric Point Distribution Model (PDM) and a 2D appearance model whose control points are defined by full perspective projections of the PDM. The advantage is that, assuming a calibrated camera, 3D metric shapes can be retrieved from single view images. Two algorithms and computationally efficient approximations are proposed, the Simultaneous Forwards Additive (SFA) and the Normalization Forwards Additive (NFA), both based on the Lucas Kanade framework. The SFA algorithm is computationally expensive but more accurate. It searches for shape and appearance parameters simultaneously whereas the NFA projects out the appearance from the error image and searches only for the shape parameters. Expanded solutions for the SFA and NFA are also proposed in order to take into account head self occlusions. An extensive performance evaluation is presented. The frequency of convergence for the SFA, NFA and their efficient approximation is evaluated, showing that the 2.5D model can outperform 2D based methods. The Robust extensions to occlusion were tested on a synthetic sequence showing that the model can deal robustly with large head rotation.