Reproduce “Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ali Ahmadi, Hamed Rahimi, Adrien Jacquet Cretides, Marie Samson, Mahdi Khoramshahi, Mohamed Chetouani
The HUMA framework, developed by Sorbonne Université, enables socially compliant robot navigation by dynamically switching between an efficient RL policy and a Vision-Language Model only when robots are in close proximity to humans. This approach yielded up to a 20% increase in navigation success rates and a 20% reduction in human collisions on the Social-MP3D benchmark, validating its real-time social intelligence.
Key insights
- Selectively engaging computationally intensive VLM reasoning only when socially critical situations arise effectively balances real-time operational efficiency with deep semantic understanding for social navigation.
- Converting raw depth maps to a JET colormap with Canny edge detection and goal vector overlay for VLM input significantly improves performance (SR, SPL) and reduces inference latency compared to RGB or mixed inputs.
- An intermediate proximity threshold (e.g., 0.80 m for Personal Space Compliance) for activating the VLM provides an optimal balance, ensuring timely intervention without excessive computational overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.