Image‐based surface deformation for multi‐view three‐dimensional facial reconstruction

The authors present a new technique to generate highly detailed three‐dimensional facial models from the images of a human face. Unlike the traditional stereo matching‐based multi‐view geometry and statistical model‐based approaches, the key idea of the proposed approach is to deform a template model using photometric consistency metric defined by all input images. The image‐based facial deformation as a continuous optimisation problem is formulated. The objective function ensures the deformed surface model not only matches the image measurement among all the input images but also keeps the fine details of the template model. Also, an efficient gradient‐based optimisation technique is employed to find an optimal facial model that precisely matches an individual's face in input images. Template‐based deformation avoids the difficulty because of occlusion in images, and the robust estimation technique also improve the quality of facial reconstruction. The experimental results using synthetic and real images show that the proposed method provides a stable and quality facial reconstruction on wide‐baseline images.