Reproduce “SIR: Structured Image Representations for Explainable Robot Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Paul Mattes, Jan Schwab, Jens Bosch, Nils Blank, Maximilian Xiling Li, Minh-Trung Tang, Moritz Haberland, Rudolf Lioutikov
Researchers at the Intuitive Robots Lab, Karlsruhe Institute of Technology, developed Structured Image Representations (SIR) using Scene Graphs (SGs) to enable intrinsically explainable robot learning. This approach achieved an average success rate of 19.5% on the RoboCasa benchmark, outperforming image-based baselines at 14.81%, while demonstrating robustness to distractor objects and uncovering latent dataset biases through sparse, learned sub-graphs.
Key insights
- Learning sparse, task-relevant sub-graphs provides intrinsic explainability, directly revealing the model's understanding of the scene and decision-making criteria.
- Analyzing the learned sub-graphs effectively uncovers latent dataset biases, such as spurious correlations and positional biases, which influence robot behavior.
- Structured Scene Graph representations significantly enhance model robustness to novel distractor objects, unlike traditional image embeddings which show performance degradation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.