Enhancing Temporal Coherence in Image-to-Video Facial Expression Synthesis: A Dual-Loss Framework for Smoother Generation

Facial expression synthesis for video sequences presents significant challenges in terms of maintaining temporal coherence while preserving identity and expression accuracy. Most current approaches primarily focus on individual frame quality. This results in videos with unnatural transitions, flickering artifacts, and inconsistent expression dynamics. This paper presents a novel method for improving temporal coherence in facial expression image-to-video synthesis. The proposed approach uses a dual-focus framework that explicitly models both pixel-level transitions and geometric consistency between consecutive frames. Our method includes two specialized loss functions: a frame difference loss and a landmark consistency loss. We also implement a two-stage training strategy and a frame-blending post-processing technique. Extensive testing on the MUG Facial Expression Database demonstrates that the enhanced-coherence approach outperforms baseline methods and competes with other approaches from the literature. The proposed method has the lowest Average Content Distance score of all the approaches tested. This indicates smoother frame transitions while maintaining high visual quality. Subjective evaluation with 140 participants confirms that the synthesized expressions are more realistic and expressive. Our approach outperformed real videos in recognition rates for certain emotion expressions, particularly disgust and sadness. These findings demonstrate the effectiveness of explicitly modeling temporal coherence in generating realistic facial expression videos. This has promising applications in therapy, entertainment, and human-computer interaction.