Reproduce “UniIntervene: Agentic Intervention for Efficient Real-World Reinforcement Learning” on OpenResearch

About this paper

Haoyuan Deng, Yitong Gao, Yudong Lin, Haichao Liu, Zhenyu Wu, Ziwei Wang

UniIntervene, an agentic intervention model from Nanyang Technological University, enables robotic policies to autonomously detect and recover from unproductive exploration in real-world reinforcement learning. The system reduces human intervention by 57% and improves the average success rate by 8.6% across diverse manipulation tasks compared to a state-of-the-art human-in-the-loop RL baseline.

Key insights

• An internal agentic intervention mechanism can significantly reduce human burden in real-world RL by proactively identifying and correcting unproductive rollouts.
• The temporal trend of estimated value provides a more robust signal for intervention than instantaneous value drops, effectively distinguishing true stagnation from transient state changes.
• Combining a memory of high-value states with a goal-conditioned policy allows for effective, structured recovery by guiding the robot towards feasible intermediate objectives.

Summary by alphaXiv.