Reproduce “Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Geling Liu, Yunzhi Tan, Ruichao Zhong, Yuanzhen Xie, Lingchen Zhao, Qian Wang, Bo Hu, Zang Li
Solid-SQL, a pre-processing pipeline, enhances the robustness of Text-to-SQL systems against adversarial inputs by integrating enhanced schema linking and in-context learning for large language models. The system achieves 82.1% execution accuracy on the Spider dev set and demonstrates an average 11.6% improvement on perturbed benchmarks like Spider-Syn and Dr. Spider, while maintaining compatibility with diverse LLMs.
Key insights
- Robustness issues in LLM-based Text-to-SQL can be effectively addressed at the pre-processing stage through intelligent prompt engineering, circumventing the challenges of modifying the core LLM.
- Utilizing LLMs for adversarial data augmentation is an effective method to train smaller, specialized models for robust schema linking, significantly improving resilience to input variations.
- A dual-layered in-context learning strategy, combining question and SQL structural similarity for example retrieval, provides more precise guidance to LLMs on underlying query logic.
- Explicitly highlighting relevant schema elements within a comprehensive prompt, rather than just filtering, enhances fault tolerance and improves stable SQL generation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.