Reproduce “Jellyfish: A Large Language Model for Data Preprocessing” on OpenResearch

About this paper

Haochen Zhang, Yuyang Dong, Chuan Xiao, Masafumi Oyamada

Jellyfish is a suite of instruction-tuned, local, open-source Large Language Models (7B-13B parameters) developed to serve as universal solvers for diverse data preprocessing tasks. This approach enables secure, cost-efficient, and customizable data quality operations, demonstrating competitive performance with proprietary models while also offering explicit interpretability through natural language explanations.

Key insights

• Instruction tuning effectively infuses smaller, local LLMs with universal data preprocessing capabilities, allowing them to achieve performance competitive with larger, proprietary models while operating securely within a local environment.
• The injection of general and domain-specific knowledge during instruction tuning is critical for significantly enhancing model performance and improving generalization to unseen datasets and tasks across different domains.
• While incorporating reasoning data can boost interpretability and performance for smaller models, its application to larger, more capable base models requires careful consideration, as it can sometimes compromise raw DP performance if not aligned properly with the base model's inherent reasoning.

Summary by alphaXiv.