Reproduce “Natural Language Camera Movement Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuwen Tan, Joey Huang, Jin Huang, Haoxiang Li, Boqing Gong
ACaM benchmarks natural-language understanding of atomic camera movements in real and generated video. It defines 17 movement classes and fine-tunes Qwen3-VL-8B on a resampled, augmented multi-source dataset. The model reaches 74.28% average accuracy on real clips and 77.44% on synthetic clips, exceeding Gemini 3.1 Pro, but remains below human performance and covers neither compound nor long-form movement.
Key insights
- Current VLMs struggle with fundamental distinctions in camera mechanics (e.g., translation vs. rotation, zoom vs. dolly) and often misinterpret subtle or directional movements.
- A dedicated benchmark grounded in cinematography taxonomy is essential for systematically evaluating and advancing natural language camera movement understanding.
- Targeted data augmentation and sampling are critical for addressing the long-tail distribution of camera movement types in training data, significantly improving VLM performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.