Reproduce “Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Lorenzo Baraldi, Federico Cocchi, Marcella Cornia, Alessandro Nicolosi, Alessandro Nicolosi, Rita Cucchiara

A new approach, CoDE, trains a compact Vision Transformer from scratch using contrastive learning with global-local similarities to effectively detect diffusion-generated deepfakes. The work also introduces the D3 dataset, enabling CoDE to achieve state-of-the-art accuracy and generalization on unseen generative models while maintaining high inference efficiency.

Key insights

  • Training a specialized embedding space from scratch with contrastive learning yields superior results for deepfake detection compared to fine-tuning general-purpose foundation models.
  • Leveraging both global and multi-scale local features through contrastive learning is crucial for effectively distinguishing real from diffusion-generated images, capturing subtle artifacts.
  • Integrating a comprehensive transformation pipeline during training significantly enhances a deepfake detector's robustness and generalization to real-world manipulations of synthetic content.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.