Reproduce “Enhancing Text-to-SQL Capabilities of Large Language Models via Domain Database Knowledge Injection” on OpenResearch

About this paper

Xingyu Ma, Xin Tian, Lingxiang Wu, Xuepeng Wang, Xueming Tang, Jinqiao Wang

Researchers from Chinese institutions introduced a phased training method to inject domain-specific database knowledge into large language models, improving their Text-to-SQL capabilities. This approach consistently enhanced both execution match and exact match metrics across various LLMs, reducing errors in table/column name generation and cell value matching, and demonstrating greater robustness to synonymous queries.

Key insights

• Explicitly modeling the semantic relationships between column names, table names, and their associated cell values effectively enhances LLM understanding of polysemous or ambiguous database terms.
• Boosting the co-occurrence frequency of column and table names through mutual prediction tasks significantly improves the LLMs' ability to generate correct database identifiers.
• A separate, upstream knowledge injection phase is necessary for optimal performance, as concurrent training of knowledge injection and Text-to-SQL tasks can degrade overall model efficacy.

Summary by alphaXiv.