Reproduce “TRACES: Temporal Recall with Contextual Embeddings for Real-Time Video Anomaly Detection” on OpenResearch

About this paper

Yousuf Ahmed Siddiqui, Sufiyaan Usmani, Umer Tariq, Jawwad Ahmed Shamsi, Muhammad Burhan Khan

The TRACES framework from FAST-NUCES, Karachi, introduces a memory-augmented pipeline for context-aware zero-shot video anomaly detection, leveraging textual context embeddings and temporal cross-attention. This approach achieved an AUC-ROC of 90.4% on UCF-Crime and 92.15% on XD-Violence datasets, setting new state-of-the-art zero-shot performance while enabling real-time inference.

Key insights

• The temporal cross-attention mechanism is crucial for effectively fusing appearance and motion, demonstrating substantial performance gains over simpler fusion methods (e.g., 3.4% AUC gain on UCF over concatenation).
• The diversity and representativeness of the contextual memory bank are more critical than its raw scale, with performance plateauing after a certain optimal size (around 200 traces).
• Integrating biologically inspired "memory traces" of anomalous and non-anomalous contexts, retrieved via cosine similarity with fused spatio-temporal embeddings, enables robust zero-shot anomaly detection.

Summary by alphaXiv.