Reproduce “Impostor: An Agent-Curated Benchmark for Realistic AIGC Manipulation Localization” on OpenResearch

About this paper

Zhenliang Li, Yutao Hu, Qixiong Wang, Wenpeng Du, Hongxiang Jiang, Jiasong Wu, Xiaolong Jiang, Jungong Han

Researchers from Southeast University, Xiaohongshu Inc., and Tsinghua University developed Impostor, an agent-curated benchmark dataset of 100,000 images for AIGC manipulation localization featuring high visual realism and diverse manipulation types from seven state-of-the-art generative models. Concurrently, they propose PANet, a two-stream deep learning framework that leverages local forensic cue extraction and explicit semantic-forensic consistency learning to achieve superior average IoU/F1 scores (0.5978/0.6603) across five benchmarks and 92.68% mean accuracy on cross-domain image-level detection.

Key insights

• A closed-loop agent framework (CraftAgent) combined with rigorous human verification can effectively generate a benchmark dataset that accurately reflects the challenges of modern, realistic AIGC manipulations, deceiving both humans and advanced LVLMs.
• Explicitly extracting and integrating fine-grained local forensic cues, particularly local phase structures, alongside traditional residual analysis, is crucial for detecting subtle AIGC manipulations.
• Supervising the consistency between high-level semantic representations and low-level forensic cues provides a powerful signal to localize manipulated regions where this natural consistency is disrupted, even when edits are semantically plausible.

Summary by alphaXiv.