Reproduce “On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chenpan Li, Thies Brockmoeller, Stefanie Jegelka, Sharut Gupta, Nikolaos Karalias

The authors show that independently reliable tasks can lose accuracy when combined in one prompt. They find composition reduces per-task thinking tokens, with longer-trace tasks more vulnerable. Measured-fragility batching cuts generated output tokens by 43% on average while changing accuracy by +0.6 percentage points; this result excludes AIME and relies on empirical fragility estimates.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.