ViT 랭크 붕괴 완화를 통한 딥페이크 탐지

초록

The recent increase in privacy violations using deepfakes has become a growing concern. Current approaches relying on ViT's global information processing face limitations in detecting localized forgery traces in deepfake images, highlighting the need for additional local feature learning. This issue is closely related to attention over-smoothing occurring within ViT. In this study, we introduce LocAt and FDAM on top of DeepShield — a baseline model that leverages spatial tokens within ViT. Our approach demonstrates notable performance improvements on DF40, while showing limitations on DFDC and CDFv2.

키워드