Reproduce “CUEing User Simulators: Calibrated User Embeddings for Multi-Turn Benchmarking” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anjali Kantharuban, Jonas Mueller
CUE turns a human interaction into instructions and examples that condition a simulator language model, using either an observed conversation or a sampled session representation; the simulator need not be fine-tuned. With Gemini 3.5 Flash Lite on τ²-Bench, CUE lowers success-rate error from 0.21 to 0.12 and raises replay failure-mode macro-F1 from 0.14 to 0.22 versus None, but results vary across settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.