Reproduce “AntifakePrompt: Prompt-Tuned Vision-Language Models are Fake Image Detectors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
You-Ming Chang, Chen Yeh, Wei-Chen Chiu, Ning Yu
National Yang Ming Chiao Tung University and Netflix Eyeline Studios present AntifakePrompt, a method that adapts Vision-Language Models (VLMs) through prompt tuning to detect fake images. This approach achieves 92.11% average accuracy across 23 diverse, unseen generative models with only 4864 trainable parameters, demonstrating superior generalizability and resource efficiency.
Key insights
- Vision-Language Models (VLMs) possess inherent zero-shot generalization capabilities that can be effectively harnessed for robust deepfake detection across diverse generators.
- Prompt tuning offers a highly parameter-efficient method to adapt large foundation models, allowing for superior performance with minimal trainable parameters and reduced data requirements.
- Formulating deepfake detection as a VQA problem allows VLMs to leverage their semantic understanding of 'real' vs. 'fake' content, moving beyond artifact-specific detection.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.