Reproduce “Prompting-MammAlps: Fine-Grained Text-to-Video Retrieval for Camera-Trap Data” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Valentin Gabeff, Baptiste Maquignaz, Jennifer Shan, Sepideh Mamooler, Gencer Sumbul, Blair Costelloe, Devis Tuia, Alexander Mathis

Researchers developed Prompting-MammAlps, the first camera-trap specific Text-to-Video Retrieval benchmark, alongside an extended MammAlps-S2 dataset. Their interpretable, LLM-guided TVR method achieved an F1-score of 0.34, outperforming traditional VLM baselines for fine-grained ecological analysis.

Key insights

  • Visual perception, specifically accurate fine-grained attribute extraction from camera-trap footage, is the primary performance bottleneck for end-to-end ecological text-to-video retrieval, rather than LLM-based query interpretation.
  • Off-the-shelf Video-Language Models demonstrate limited generalization to specialized ecological camera-trap data without domain adaptation, underscoring the need for tailored visual models.
  • Constraining an LLM with a custom library of primitive parsing functions and few-shot examples effectively enables it to generate complex, interpretable retrieval logic for fine-grained queries, mitigating hallucination.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.