Reproduce “"Don't Do That!": Guiding Embodied Systems through Large Language Model-based Constraint Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Amin Seffo, Aladin Djuhera, Masataro Asai, Holger Boche
The STPR framework enables embodied robotic systems to adhere to complex, informal natural language constraints during navigation by translating them into precise, executable Python functions. This neuro-symbolic approach, developed at Technical University Munich and MIT-IBM Watson AI Lab, integrates these functions into traditional pathfinding algorithms, achieving 100% constraint compliance in diverse simulated scenarios, from avoiding camera fields of view to dynamic conditional rules.
Key insights
- Leveraging LLMs for explicit code generation (interpretable Python functions) rather than direct plan generation effectively mitigates issues like hallucinations and partial compliance, enabling robust and guaranteed constraint adherence in robotic navigation.
- The framework's ability to handle diverse and complex constraints—including mathematical models (camera FOV), physics-based rules (heat dissipation), and conditional semantic instructions—significantly extends the scope of natural language control for robots beyond simple obstacle avoidance.
- STPR's modular design ensures interpretability and auditability of generated constraints, is compatible with cost-effective smaller code LLMs, and maintains the theoretical guarantees of classical pathfinding algorithms, enhancing safety and transparency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.