Reproduce “An Automatic Prompt Generation System for Tabular Data Tasks” on OpenResearch

About this paper

Ashlesha Akella, Abhijit Manatkar, Brij Chavda, Hima Patel

An automatic prompt generation system for tabular data tasks is introduced, leveraging a Reinforcement Learning agent for optimal column selection and sequencing, and a Cell-Level Similarity measure for few-shot example selection. This system significantly improves the performance of large language models (LLMs) on data imputation, error detection, and entity matching tasks, demonstrating gains of up to 41 percentage points in accuracy on various benchmarks.

Key insights

• Automating column selection and sequencing using Reinforcement Learning significantly boosts LLM performance for tabular tasks, overcoming the limitations of manual or heuristic approaches.
• Preserving the semantic content of individual cells for few-shot example selection, rather than serializing rows into sentences, leads to more effective in-context learning for structured tabular data.
• The comprehensive prompt generation approach (RLCS-CLFS) is broadly applicable and robust, improving results across diverse tabular datasets and tasks like data imputation, error detection, and entity matching.

Summary by alphaXiv.