Reproduce “May the Dance be with You: Dance Generation Framework for Non-Humanoids” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hyemin Ahn
This research from UNIST introduces a framework allowing non-humanoid agents to learn how to dance from human dance videos, bypassing the need for robot-specific data. The method trains a reward model on human optical flow and music, then uses it to guide reinforcement learning for diverse non-humanoid agents like CartPole and a UR5 robot, producing movements highly favored by human evaluators.
Key insights
- Dance can be defined abstractly as a motion forming a visual rhythm perceivable from optical flow synchronized with music, allowing generalization across different agent anatomies.
- A reward model trained on human dance data effectively guides non-humanoid agents, decoupling their learning process from direct, synchronized human video input.
- Learning-based dance generation for non-humanoids is achievable by leveraging existing human dance datasets and focusing on the underlying visual rhythm-music relationship.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.