Reproduce “BAP v2: An Enhanced Task Framework for Instruction Following in Minecraft Dialogues” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Prashant Jayannavar, Liliang Ren, Marisa Hudspeth, Risham Sidhu, Charlotte Lambert, Ariel Cordes, Elizabeth Kaplan, Anjali Narayan-Chen and 1 more
This paper presents BAP v2, an improved framework for grounded instruction following within Minecraft dialogues, enhancing evaluation, data generation, and modeling. Researchers at the University of Illinois Urbana-Champaign developed the Llama-CRAFTS model, which combines richer input representations with synthetically generated data, achieving a new state-of-the-art F1 score of 53.0 on the human-generated test set.
Key insights
- Spatial reasoning, particularly the ability to predict precise 3D locations, is the primary bottleneck for model performance in grounded instruction following, as quantitatively identified by the Location F1 metric.
- A synergistic approach, combining targeted synthetic data with explicit, richer input representations (e.g., Builder's POV and world state), is crucial for significantly advancing LLM capabilities in embodied tasks.
- Training models solely on complex human-generated dialogues is insufficient; structured synthetic data is essential to teach fundamental spatial concepts comprehensively and improve robustness on basic spatial reasoning challenges.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.