Reproduce “Detecting AI-Generated Images via Distributional Deviations from Real Images” on OpenResearch

About this paper

Yakun Niu, Yingjian Chen, Lei Zhang

Researchers at Henan University developed a Masking-based Pre-trained model Fine-Tuning (MPFT) strategy to detect AI-generated images by identifying distributional deviations from real images. This approach achieved 98.2% average accuracy on the GenImage benchmark and 94.6% on UniversalFakeDetect, demonstrating enhanced generalization to images from unseen generative models compared to direct fine-tuning.

Key insights

• Pre-trained CLIP-ViT excels at representing and clustering real images in feature space, with AI-generated images often appearing as outliers due to a distribution shift.
• Directly fine-tuning CLIP-ViT on detection tasks commonly results in overfitting to specific, low-level forgery artifacts, hindering its ability to generalize across different generative models.
• Explicitly masking texture-rich areas during training prevents the model from exploiting these artifact-prone regions, compelling it to learn more robust and universal indicators of artificiality.

Summary by alphaXiv.