Reproduce “KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyu Shao, Keru Zhou, Guowei Huang, Yajun Gao, Tongtong Cao, Xiu Li
A framework extracts kinematic affordance maps from a frozen image-to-video model, offering both localization and directional cues for robot manipulation. This method enhances policy learning from limited demonstrations by providing a computationally efficient first-order visual prior that improves success rates on tasks requiring specific kinematic interactions.
Key insights
- A frozen, general-purpose image-to-video generative model can implicitly encode valuable 'first-order' directional kinematic priors in its latent space when queried once at the noise endpoint.
- Efficiently compressing this dense latent velocity field into a small set of tokens allows for effective integration of this prior into a downstream policy without substantial computational overhead.
- Leveraging this single-query latent prior provides a computationally efficient method to enhance robot manipulation policies with sophisticated temporal knowledge from foundation models, particularly in low-data regimes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.