Reproduce “Blindfold Baselines for Embodied QA” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ankesh Anand, Eugene Belilovsky, Kyle Kastner, Hugo Larochelle, Aaron Courville

The authors test whether EmbodiedQA rewards environmental understanding by comparing navigation-and-vision systems with question-only baselines that never observe simulated homes. The BoW baseline averages word embeddings and predicts answer categories. On EQAv1 test top-1 accuracy, it scores 50.34% at T50, versus 44.32% for NMC (BC+A3C); blindfold methods lead except at T10. This comparison does not establish why systems differ.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.