Reproduce “PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Zhangyue Yin, Zixian Huang, Hanxu Hu, Jing Xiong and 9 more
PrismaForge synthesizes task-specific rubrics to evaluate programs for data selection and reinforcement-learning training. Each rubric checks source code and rendered output separately against requirements tailored to an instruction. In equal-sized filtering, rubric-selected data scored 72.6 versus 48.5 on Vega-Lite and scored higher across all eight benchmarks. This supports sample selection under those conditions, not a full reinforcement-learning comparison.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.