Reproduce “Formally Exploring Time-Series Anomaly Detection Evaluation Metrics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dennis Wagner, Arjun Nair, Billy Joe Franks, Justus Arweiler, Aparna Muraleedharan, Indra Jungjohann, Fabian Hartung, Mayank C. Ahuja and 19 more
The authors develop and analyze evaluation metrics for time-series anomaly detection (TSAD), specifying nine basic properties that scores should satisfy. They formally assess 37 existing metrics and find none meets all nine. They prove LARM satisfies these properties and ALARM a stricter set; an SMD8 example shows rankings can change with metric choice, not which detector is best.
Key insights
- A systematic analysis of 37 widely used TSAD evaluation metrics revealed that none of them satisfy all nine proposed formal properties, explaining prior inconsistencies.
- Many existing metrics struggle with adequately penalizing redundant alarms, false positives, and false alarms, or inherently ignore crucial temporal information.
- The introduced ALARM metric, with its configurable 'alarm tolerance' parameter, offers flexible control over application-specific trade-offs between detection benefits and false alarm costs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.