Reproduce “Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tomáš Souček, Jean-Baptiste Alayrac, Antoine Miech, Ivan Laptev, Josef Sivic

The authors present a self-supervised method to locate object states and actions in untrimmed videos. It uses initial-state, action, and end-state order as a learning constraint, while exemplar images weight videos by relevance. On ChangeIt, it achieves 0.35 state precision and 0.68 action precision, versus 0.30 and 0.63 for zero-shot CLIP ViT-L/14; these are localization results on annotated videos, not evidence of physical causality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.