Premium
Real‐Time Facial Expression Transformation for Monocular RGB Video
Author(s) -
Ma L.,
Deng Z.
Publication year - 2019
Publication title -
computer graphics forum
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.578
H-Index - 120
eISSN - 1467-8659
pISSN - 0167-7055
DOI - 10.1111/cgf.13586
Subject(s) - computer science , facial expression , artificial intelligence , computer vision , expression (computer science) , monocular , transformation (genetics) , face (sociological concept) , rgb color model , computer graphics (images) , social science , biochemistry , chemistry , sociology , gene , programming language
This paper describes a novel real‐time end‐to‐end system for facial expression transformation, without the need of any driving source. Its core idea is to directly generate desired and photo‐realistic facial expressions on top of input monocular RGB video. Specifically, an unpaired learning framework is developed to learn the mapping between any two facial expressions in the facial blendshape space. Then, it automatically transforms the source expression in an input video clip to a specified target expression through the combination of automated 3D face construction, the learned bi‐directional expression mapping and automated lip correction. It can be applied to new users without additional training. Its effectiveness is demonstrated through many experiments on faces from live and online video, with different identities, ages, speeches and expressions.