Reproduce “Leveraging Failed Samples: A Few-Shot and Training-Free Framework for Generalized Deepfake Detection” on OpenResearch

About this paper

Shibo Yao, Renshuai Tao, Xiaolong Zheng, Chao Liang, Chunjie Zhang

The Few-shot Training-free Network (FTNet) and its fine-tuned variant (FTNet-T) enable generalized deepfake detection by leveraging a minimal number of target domain samples from a pre-trained vision-language model. This framework achieved state-of-the-art performance, with FTNet-T improving average accuracy by 12.1% over existing methods, especially in cross-domain and few-shot scenarios.

Key insights

• Effectively leveraging a very small number of newly encountered deepfake samples, even those previously misclassified, can significantly improve detection capabilities without extensive retraining.
• Intermediate features from powerful pre-trained vision-language models, such as CLIP's L12 layer, contain universal forgery signatures that are critical for distinguishing real from fake, even across diverse semantic content.
• Framing deepfake detection as a few-shot classification problem offers a more practical and adaptable solution compared to resource-intensive zero-shot generalization or large-scale retraining.
• The approach of using a dynamic, memory-based cache for classification enables rapid adaptation to novel deepfake types with minimal computational overhead.

Summary by alphaXiv.