Bias-Averse Learning for Mitigating Source Dataset Bias in Avatar Fingerprinting

초록

Talking-head generation produces avatars from a source (the driver) and a target (the identity). In the context media forensics, standard deepfake detection can only distinguish between "real" vs. "fake" content, identifying the specific driver for privacy purposes remains a less explored task. Some prior work has already addressed this task, called 'avatar fingerprinting'. However, existing methods face two significant problems: 1) The proposed model contained strong dataset bias. 2) The evaluation dataset failed to penalizes those biases. Based on our investigation, the dataset bias occurred due to a lacks of pixel information. As a result, it suffers from poor performance in same driver dataset settings. To this end, we introduce a bias-averse learning framework. We design a method to learn a dynamic signature from pixel-level dynamic information and use a domain orthogonality loss that drives the learned features to be data-agnostic, introducing data bias free. Then, we introduce a more rigorous 'single-source' evaluation protocol that reveals the reliance on shortcuts in prior methods. Our experiments show that our method outperforms prior work by up to 20% in both area under the curve (AUC) and average precision (AP) with this protocol.

키워드