Reproduce “Humanoid Locomotion as Next Token Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, Jitendra Malik
Researchers at UC Berkeley developed a method for humanoid locomotion by reframing it as a next token prediction task, similar to large language models. This approach uses a transformer model trained on diverse datasets to generate robust locomotion policies, achieving zero-shot transfer to a full-sized humanoid robot that navigated varied terrains in San Francisco.
Key insights
- Humanoid locomotion can be effectively modeled as a sequence prediction problem, analogous to natural language processing.
- Transformer architectures can capture complex sensorimotor patterns and long-range dependencies, enabling learning from both complete and incomplete training data.
- Performance scales with increased training data, context length, and model size, similar to observations in large language models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.