Reproduce “Memory-Anchored Multimodal Reasoning for Explainable Video Forensics” on OpenResearch

About this paper

Chen Chen, Runze Li, Zejun Zhang, Pukun Zhao, Fanqing Zhou, Longxiang Wang, Haojian Huang

The paper introduces FakeHunter, an explainable multimodal framework for deepfake detection that combines memory-guided retrieval, structured Chain-of-Thought reasoning, and adaptive tool-augmented verification. It also presents X-AVFake, a comprehensive dataset with multimodal manipulations and fine-grained annotations for training and evaluating explainable detectors, achieving an overall accuracy of 34.75% on this new benchmark while providing human-readable justifications.

Key insights

• Combining memory-guided retrieval with adaptive tool-augmented verification within a structured VLM-based reasoning pipeline significantly enhances multimodal deepfake detection accuracy and interpretability by providing contextual grounding and localized evidence.
• Explicit Chain-of-Thought reasoning, through an Observation-Thought-Action sequence, is essential for decomposing complex deepfake analysis into human-understandable steps and generating detailed explanations.
• The adaptive invocation of specialized tools (e.g., visual zoom-in, audio spectrogram analysis) for fine-grained scrutiny improves detection confidence, particularly in cases where initial VLM assessments are uncertain or for uncovering subtle artifacts.
• The development of comprehensive datasets with multimodal manipulations and detailed explainability annotations, like X-AVFake, is crucial for advancing the training and rigorous evaluation of explainable deepfake detection models.

Summary by alphaXiv.