Reproduce “RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Siyuan Ma, Yu Qi, Cheng Ye, Ruotian Peng, Xinyi Xu, Zitiantao Lin, Zedong Cai, Yijian Huang and 3 more
RoboJudge evaluates video rollouts by scoring Physical Adherence (PA), whether interactions remain physically plausible, separately from Instruction Alignment (IA), whether the requested action and goal are achieved. RoboJudge-9B is fine-tuned on human-verified rubric scores. On 800 benchmark videos, its correlation with human ratings is 0.719 versus 0.630 for Gemini-3.7-Flash; this measures agreement, not physical truth or robot safety.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.