Reproduce “RoboJev: An Open Vision-Language Judge for Robot Success Detection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Josef Liyanjun Chen

RoboJev-27B judges whether robot episodes complete their instructions by comparing task text with ten sampled video frames and, when available, final auxiliary-camera views, then returning a calibrated success probability. It adapts JEV-27B-VL with rank-32 LoRA adapters. On 268 held-out SO-101 episodes, it reached 95.2% balanced accuracy versus 88.1% for the matched-input base; unseen-domain transfer improvement was not established.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.