Reproduce “AnomalyGen: Enhancing Log-Based Anomaly Detection with Code-Guided Data Augmentation” on OpenResearch

About this paper

Xinyu Li, Yintong Huo, Chenxi Mao, Shiwen Shan, Yuxin Su, Yanlin Wang, Zibin Zheng

Addressing data sparsity in log-based anomaly detection, AnomalyGen synthesizes labeled log sequences from source code, leveraging static analysis and large language models for augmentation. This method consistently improved F1-scores for deep learning anomaly detection models by up to 15.2% on benchmark datasets, leading to fewer false alarms.

Key insights

• An empirical study quantified significant data sparsity in public log anomaly detection benchmarks (e.g., HDFS 0.99% coverage), highlighting a critical data limitation rather than solely a modeling one.
• LLM-based Chain-of-Thought verification is the most critical component, ensuring logical consistency and realistic parameter values, with its removal leading to a 10.7 percentage point drop in F1-score for some models.
• Unsupervised deep learning models, particularly those using sequence-aware encodings, benefited most from AnomalyGen, achieving substantial F1-score gains due to the broader coverage of structurally valid normal execution paths.

Summary by alphaXiv.