Reproduce “LLM-Enhanced Reinforcement Learning for Time Series Anomaly Detection” on OpenResearch

About this paper

Bahareh Golchin, Banafsheh Rekabdar, Danielle Justo

A framework enhances time series anomaly detection by integrating Large Language Models (LLMs) for semantic reward shaping into a Reinforcement Learning system, complemented by Variational Autoencoders and active learning. This approach achieved F1-scores of 0.7413 on Yahoo-A1 and 0.5300 on SMD datasets, demonstrating robust performance under limited labeling conditions.

Key insights

• LLM-based reward shaping effectively guides RL agents, with Llama-3.2-3B proving most effective by producing balanced and consistent anomaly severity scores.
• Different LLM architectures exhibit distinct biases in assessing anomaly likelihood, with GPT-3.5 being 'over-eager' and Phi-2 being 'conservative', while Llama-3 provides a robust balance.
• The dynamic integration of unsupervised VAE signals, supervised RL feedback, and active learning provides a data-efficient and robust solution for complex time series anomaly detection.

Summary by alphaXiv.