Reproduce “Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaboration” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuxiang He, Jian Zhao, Yuchen Yuan, Tianle Zhang, Wei Cai, Haojie Cheng, Ziyan Shi, Ming Zhu and 3 more

Researchers at the Institute of Artificial Intelligence (TeleAI), China Telecom, introduced Aetheria, a multimodal content safety framework that leverages a multi-agent debate and collaboration architecture to detect implicit risks in digital content. The system achieved an F1 score of 0.84 on multimodal inputs on the newly developed AIR-Bench dataset and produces interpretable, traceable audit reports for moderation decisions.

Key insights

  • An adversarial multi-agent debate, particularly with a balanced "Strict" and "Loose" perspective, is crucial for robustly identifying implicit risks and mitigating single-model biases in complex content.
  • Integrating a RAG module to provide external context and historical cases significantly enhances the system's ability to ground decisions and improve recall for implicit risk recognition, especially in multimodal scenarios.
  • Interpretability in content moderation can be achieved through a structured debate process and hierarchical adjudication, yielding detailed audit trails and justifications for decisions, and proving cost-effective for high-stakes auditing compared to manual review.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.