Reproduce “Consider, Then Choose: Order-Invariant Decisions in System One Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jianguo Huang, Yao Long Teng, Yanchen Deng, Bo An
A duplicate-charge report must be routed to an intent, though list order may be arbitrary. Runoff scores options individually, keeps plausible candidates, then compares them across positions. On two deterministic open models, it returned unchanged answers under one shuffle across five benchmarks; on 20-option tasks, mean accuracy was 73.8%, versus 74.5% for cyclic averaging. This assumes deterministic responses and unique pointwise scores.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.