Reproduce “Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shuyu Yang, Yaxiong Wang, Li Zhu, Zhedong Zheng

A new benchmark, Pedestrian Anomaly Behavior (PAB), and a Cross-Modal Pose-aware (CMP) framework are introduced for text-based person anomaly search, enabling retrieval of individuals by appearance and detailed actions, including anomalous behaviors. The CMP framework achieved 84.93% Recall@1 on the PAB dataset and demonstrated robustness across diverse conditions.

Key insights

  • Explicitly integrating human pose information through a pose-aware image encoder significantly enhances a model's ability to discriminate between subtle behavioral differences.
  • Identity-based hard negative mining, constructing negative pairs with the same person performing an opposite action, is crucial for training models to distinguish fine-grained behaviors.
  • Large-scale synthetic data, when carefully generated and re-captioned, effectively mitigates data scarcity for anomalous behaviors and improves model performance for text-based anomaly search.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.