Reproduce “IMFD: End-to-end Multi-Face Forgery Detection through Instruction-based Large Vision-Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dasom Choi, Sangjun Moon, Hyeongchan Im, Jaeeon Park, Jingun Kwon, Hidetaka Kamigaito, Taro Watanabe, Manabu Okumura
IMFD detects forged faces in multi-face images by placing face coordinates in a vision-language model’s instruction, linking each face to its label. Its single-stage system uses predicted boxes and outperforms COMICS on OpenForensics’ ALL evaluation: micro-F1 is 0.90 versus 0.81, and exact-match accuracy is 0.77 versus 0.40. Results are limited to this dataset; localization errors affect detection.
Key insights
- Spatial cues in the prompt matter: including coordinates, image dimensions, and the number of people yields the strongest overall balance of classification metrics in the reported ablations.
- Localization is a key bottleneck: performance drops when the system uses predicted boxes instead of ground-truth coordinates, despite face-box average precision of 81.9 on the full test set.
- Performance is not uniformly better across all settings: IMFD beats COMICS on the full test set, but on the LARGE subset its micro F1 is lower.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.