Reproduce “Human Action CLIPs: Detecting AI-generated Human Motion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Matyas Bohacek, Hany Farid
Researchers from Google, Stanford, and UC Berkeley propose a robust method for detecting AI-generated human motion in videos by using multi-modal semantic embeddings. Their approach, leveraging fine-tuned CLIP models, achieves up to 99.1% video-level accuracy and demonstrates resilience against common video laundering techniques, while also generalizing well to unseen generative models.
Key insights
- Multi-modal semantic embeddings are highly effective for detecting AI-generated human motion, exhibiting superior robustness to video laundering and better generalization to unseen generative models compared to low-level feature methods.
- Fine-tuning a generic CLIP model (FT-CLIP) significantly boosts detection performance, achieving the highest accuracy for supervised classification tasks.
- An unsupervised "frame-to-prompt" method offers competitive performance without requiring explicit training on labeled deepfake data, simplifying deployment and enhancing generalizability to new generative models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.