Reproduce “Be My Eye: Learning to See Beyond a Robot's Own Perspective” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Baifu Huang, Shilin Fang, Jianan Wang
Be My Eye shares compact task-conditioned context between robots, allowing a receiver to combine a partner’s view with its own. A query-based encoder compresses features and task instructions into tokens, refined through receiver-side action supervision. In occluded-target selection, Ours-Joint achieves 88.9% accuracy versus 72.2% for Generic SigLIP, using 32 rather than 160 tokens; the stationary-provider test does not assess coordinated manipulation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.