Reproduce “OutfitterBench: Evaluating LLM Agents on Long-Horizon Resource Allocation and Commitment Coordination” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jie Huang, Yuecheng Hong, Yao Chen, Shengding Jiao, Xudong Liu, Qiang Ye

OutfitterBench evaluates long-running rental agents on coordinating bookings for reusable equipment with intervening rentals, returns, cleaning, and uncertain demand. It compares seven model configurations with a fixed Calendar policy across two simulated worlds. Only GPT 6.1 Sol exceeds Calendar in both; mean value creation is +2,556.78 CNY versus +1,119.51 CNY. Results are limited by two seeds and rule-based simulated customers.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.