Reproduce “PET-SQL: A Prompt-Enhanced Two-Round Refinement of Text-to-SQL with Cross-consistency” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhishuai Li, Xiang Wang, Jingjing Zhao, Sun Yang, Guoqing Du, Xiaoru Hu, Bin Zhang, Yuxiao Ye and 3 more
PET-SQL is a two-round framework that refines Text-to-SQL generation by introducing a SQL-Tailored Prompt, PreSQL-based schema linking, and a cross-consistency mechanism. This approach, developed by SenseTime Research and collaborators, achieves a new state-of-the-art execution accuracy of 87.6% on the challenging Spider benchmark.
Key insights
- Explicitly providing cell value references in prompts is highly effective in mitigating common data formatting errors, improving SQL execution accuracy.
- Leveraging LLMs' strength in code generation to infer relevant schema from a preliminary SQL query (PreSQL) is more effective and efficient than direct schema linking instructions, significantly reducing prompt length.
- Ensembling diverse LLMs with stable, low-temperature generations (Cross-consistency) leads to substantially higher gains in SQL refinement compared to single-model self-consistency, by mitigating errors and leveraging varied model strengths.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.