Reproduce “RoboJev: An Open Vision-Language Judge for Robot Success Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Josef Liyanjun Chen
RoboJev-27B judges whether robot episodes complete their instructions by comparing task text with ten sampled video frames and, when available, final auxiliary-camera views, then returning a calibrated success probability. It adapts JEV-27B-VL with rank-32 LoRA adapters. On 268 held-out SO-101 episodes, it reached 95.2% balanced accuracy versus 88.1% for the matched-input base; unseen-domain transfer improvement was not established.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.