Reproduce “Inference-Time Policy Steering through Human Interactions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanwei Wang, Lirui Wang, Yilun Du, Balakumar Sundaralingam, Xuning Yang, Yu-Wei Chao, Claudia Perez-D'Arpino, Dieter Fox and 1 more
Researchers from MIT CSAIL and NVIDIA developed Inference-Time Policy Steering (ITPS), a framework that allows real-time human guidance of pre-trained generalist robotic policies. Their proposed Stochastic Sampling (SS) method effectively aligns policy outputs with diverse human intents while preserving learned physical constraints, demonstrated by achieving 71% task alignment and 73% constraint satisfaction in a real-world kitchen rearrangement task.
Key insights
- A fundamental trade-off exists between improving alignment with human intent and maintaining constraint satisfaction when steering generative policies.
- Multimodal generative policies, such as Diffusion Policy, are more amenable to real-time steering compared to unimodal policies like ACT.
- The proposed Stochastic Sampling (SS) method consistently achieves a superior balance between aligning with human intent and preserving the learned data manifold (constraint satisfaction), outperforming other steering methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.