Reproduce “Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Harshita Chopra, Kshitish Ghate, Aylin Caliskan, Tadayoshi Kohno, Chirag Shah, Natasha Jaques
Persona Policies (PPol) evolves instructions that vary simulated users’ communication while preserving task goals. It generates persona-conditioned interactions and optimizes them against human conversation references using behavioral fingerprints. In τ²-bench Retail, blinded raters judged PPol users human in 80.4% of ratings, near 80.0% for real users; this measures perceived human-likeness, not the full range of human behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.