Controllable Facial Expression Synthesis via implicit keypoints and continuous labels

초록

Facial Expression Synthesis (FES) is important for applications in virtual reality, human-computer interaction, and digital media. While recent facial animation methods have shown strong visual quality under image-driven settings, continuous-label-driven FES remains challenging because facial expressions must be generated from compact semantic conditions without paired target images, making the problem inherently under-constrained. In this paper, we propose VAPortraits, a framework for controllable facial expression synthesis that manipulates implicit keypoints through an Expression Adjustment Network (EAN). Rather than relying on driving-image supervision, our method learns label-conditioned updates in the implicit keypoint space while preserving the strong generative prior of a fixed keypoint-based synthesis backbone. We consider two continuous-label representations: valence-arousal (VA) and valence-eye-lip (VEL), where the latter replaces arousal with explicit eye and lip openness ratios for more practical control of salient facial motions. To alleviate the trade-off between semantic controllability and identity preservation in this regime, we further introduce a selective regularization loss on statistically identified dead elements. Extensive experiments demonstrate that VAPortraits achieves strong expression accuracy, identity retention, and visual quality, providing an effective solution for realistic and controllable facial expression synthesis under low-dimensional continuous conditions.

키워드