Reproduce “Text-to-SQL Models Make Choices Without Saying So” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yichen Xu, Shipei Lin, Chenhao Ma
SpecBridge tests whether text-to-SQL systems compute alternative readings of open analytical requests and report them. It pairs an open request with two identical requests specifying each interpretation, scoring SQL by execution and explanations separately. On 204 reviewed request sets, models answered specific requests accurately but often omitted readings they could compute; these results cover choices, not all SQL errors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.