Reproduce “MLLM-Enhanced Face Forgery Detection: A Vision-Language Fusion Solution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siran Peng, Zipei Wang, Li Gao, Xiangyu Zhu, Tianshuo Zhang, Ajian Liu, Haoyuan Zhang, Zhen Lei
Researchers at the Chinese Academy of Sciences Institute of Automation developed VLF-FFD, a vision-language fusion framework for face forgery detection that integrates visual and textual information to achieve state-of-the-art performance and explainability. The framework yielded an AUC of 97.29% on the challenging Celeb-DeepFake-v2 dataset and 99.91% on the FF++ benchmark.
Key insights
- Truly bidirectional vision-language fusion is essential for achieving superior deepfake detection performance and generating precise, aligned explanations.
- Developing explainability-driven datasets with frame-level, contrastive annotations significantly improves MLLM's ability to identify and describe forgery artifacts.
- A multi-stage training approach optimizes the integration of robust visual features from an external detector with the explanatory power of an MLLM.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.