Reproduce “SAGE: An LLM-driven Self Reflective Agentic Framework for Fraud Detection” on OpenResearch

About this paper

Yichen Chen, Siying Li, Yuhang Liang, Lijun Wang, Renyang Liu

SAGE, an LLM-driven self-reflective agentic framework, automates the construction of accurate, robust, and interpretable fraud detectors for individual-level tabular data. The framework achieved a 96.00% win rate against five baselines, including AutoML systems and human experts, across diverse datasets and demonstrated significant performance improvements, such as a 135.43% F1 increase on the PaySim dataset.

Key insights

• The Data Diagnostic Tree (DDT) effectively condenses raw dataset information into a structured, semantically annotated format, enabling LLMs to develop a global and fraud-aware understanding of complex tabular datasets.
• Casting the iterative model optimization as an MDP, combined with natural-language gradients for critique and a composite, fraud-specific reward function, allows for principled and interpretable model refinement under real-world constraints.
• A self-reflective loop, where the Optimization Agent utilizes a trajectory history of past code, outcomes, and diagnoses, enables it to learn and adapt its strategy for subsequent code edits.

Summary by alphaXiv.