Reproduce “PersonaCodeBench: Evaluating Coding Agents in Persona-driven Human-Interactive Sessions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haixin Wang, Jianheng Hou, Hongyu Shen, Wenzhe Fan, Wei-Chieh Huang, Qianyu Zhu, Weichen Li, Taoran Li and 22 more
PersonaCodeBench evaluates coding agents on both native software tasks and explicit user requirements. It binds persona-specific needs to checkable obligations, reveals them during interaction, and separately verifies task completion and personalization. On its 100-task Lite benchmark, GPT-6 Astra at max effort achieved a 48% Joint pass rate. This counts non-recorded passes as zero, and does not measure human satisfaction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.