Reproduce “Market-Bench: Benchmarking Large Language Models on Economic and Trade Competition” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yushuo Zheng, Huiyu Duan, Zicheng Zhang, Yucheng Zhu, Xiongkuo Min, Guangtao Zhai
Market-Bench tests how language and business decisions shape outcomes in a simulated retail market. It couples inventory auctions and pricing with slogans that influence which buyers consider each seller. Across 10 default-setup runs, Gemini 2.5 Pro earned 36,589 cumulative profit versus Phi-4’s 7,565, despite similar slogan-persona match scores. This result applies to the tested configuration and does not show slogans caused profit differences.
Key insights
- Successful selling requires more than persuasive language: agents must secure inventory, manage costs, and price goods effectively under competition.
- Profit was strongly associated with fill rate (Spearman ρ = 0.88) and bid efficiency (ρ = 0.68), while slogan-persona alignment had only a weak association with profit (ρ = 0.16).
- Performance differences can compound: early procurement success may support future bids, while agents that acquire little stock can lose ground; aggregate inequality rose during the simulated episodes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.