Reproduce “Contextual Safety Reasoning and Grounding for Open-World Robots” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zachary Ravichandran, David Snyder, Alexander Robey, Hamed Hassani, Vijay Kumar, George J. Pappas

The CORE framework enables robots to infer, ground, and enforce context-dependent safety rules directly from visual observations in dynamic, open-world environments, without requiring prior knowledge or user-specified constraints. It integrates Vision-Language Models for semantic reasoning with Control Barrier Functions for probabilistic safety guarantees, achieving 95.0% success in diverse simulations and 86.6% on a Spot robot in real-world tasks.

Key insights

  • Vision-Language Models can effectively perform online contextual safety reasoning from visual observations, especially when guided by structured prompts including defined safety logic and chain-of-thought justifications.
  • Semantic safety constraints inferred from VLMs can be robustly grounded into spatially-defined safe sets (via signed distance fields) without reliance on prior maps, making them actionable for robot control.
  • Probabilistic safety guarantees can be integrated into CBF-based control synthesis by accounting for VLM inference and semantic grounding uncertainties, enhancing confidence in robot autonomy in uncertain real-world conditions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.