Reproduce “MulVul: Retrieval-augmented Multi-Agent Code Vulnerability Detection via Cross-Model Prompt Evolution” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zihan Wu, Jie Xu, Yun Peng, Chun Yong Chong, Xiaohua Jia

MulVul, a retrieval-augmented multi-agent framework, improves code vulnerability detection by achieving a 34.79% Macro-F1 score across 130 CWE types, representing a 41.5% relative improvement over prior methods. The system effectively addresses vulnerability heterogeneity and few-shot detection challenges through specialized agents and automated prompt optimization.

Key insights

  • A hierarchical multi-agent system with selective routing can efficiently manage the complexity of multi-class vulnerability detection by activating only relevant specialized agents.
  • Automating prompt optimization through a decoupled, cross-model evolutionary process yields more robust, specialized, and generalizable instructions compared to manual or single-model methods.
  • Grounding LLM reasoning with a structured, dynamically retrieved knowledge base, particularly with contrastive examples, is crucial for mitigating hallucinations and significantly enhancing detection accuracy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.