Reproduce “Vid2Player: Controllable Video Sprites that Behave and Appear like Professional Tennis Players” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haotian Zhang, Cristobal Sciutto, Maneesh Agrawala, Kayvon Fatahalian
A system from Stanford University generates interactively controllable video sprites of professional tennis players from annotated broadcast footage, replicating their appearance and strategic behavior during rallies. It produces realistic match simulations, including player-specific styles and opponent-adaptive strategies, and supports novel applications like cross-gender matchups and point editing.
Key insights
- Integrating deep domain knowledge about the cyclic structure of tennis rallies and player decision-making significantly enhances the realism and controllability of video sprite synthesis, especially with limited training data.
- Behavioral modeling techniques typically used in sports analytics can be repurposed to drive interactive character animation, enabling sprites to not only appear but also strategically play like real athletes.
- Combining extensive offline preprocessing for visual quality (sprite matting, body completion, appearance normalization) with online, data-driven behavioral models allows for the generation of visually and behaviorally realistic human performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.