Reproduce “RedChronos: A Large Language Model-Based Log Analysis System for Insider Threat Detection in Enterprises” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chenyu Li, Zhengjia Zhu, Jiyan He, Xiu Zhang

RedChronos introduces a Large Language Model-Based Log Analysis System for Insider Threat Detection in Enterprises, employing Query-Aware Weighted Voting and a Semantic Expansion-based Genetic Algorithm with LLM-driven Mutations. The system achieved a 0.988 accuracy and 0.993 detection rate on the CERT r5.2 dataset, while reducing manual log analysis workload by approximately 90% in a real-world enterprise deployment.

Key insights

  • An ensemble approach like Query-Aware Weighted Voting can effectively mitigate the inherent inconsistencies and hallucination tendencies of individual LLMs, leading to more reliable detection in critical security applications.
  • Automating prompt optimization through a semantic expansion-based genetic algorithm allows for continuous and efficient adaptation of LLM-based systems to evolving threat landscapes without manual intervention.
  • Different LLM architectures exhibit significant performance variability even when using identical prompts, underscoring the necessity for careful model selection and robust output aggregation strategies in real-world deployments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.