Reproduce “From Noise to Signal: Improving Security Log Anomaly Detection Using LLMs with Endpoint-Specific Logs” on OpenResearch

About this paper

Christopher Henshaw, Gour Karmakar

Instruction-tuned Large Language Models (LLMs) enhance endpoint-specific authentication anomaly detection, particularly for subtle threats, by achieving higher recall and lower false negative rates compared to traditional rule-based and statistical methods. Meta Llama 3.1 8B Instruct demonstrated 88.2% recall, significantly surpassing Wazuh's 31.4% and OpenSearch's 25.5%, for instance, while consistently providing valid structured outputs.

Key insights

• Instruction-tuned LLMs can serve as an effective complementary analysis layer, providing semantic and contextual reasoning for cybersecurity log anomaly detection beyond the capabilities of rule-based and statistical systems.
• Detecting subtle "borderline" anomalous behaviors, which often evade traditional methods, is a key strength of well-performing LLMs like Meta Llama 3.1 8B Instruct.
• Operational aspects, including inference latency and consistent structured-response validity, are critical for the practical deployment and integration of LLMs in automated security workflows.

Summary by alphaXiv.