Reproduce “Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes” on OpenResearch

About this paper

Simran Arora, Brandon Yang, Sabri Eyuboglu, Avanika Narayan, Andrew Hojel, Immanuel Trummer, Christopher Ré

Researchers at Stanford University and Cornell University developed `Evaporate`, a system leveraging Large Language Models (LLMs) to automatically convert diverse, semi-structured data lakes into queryable, structured tables. Its advanced implementation, `Evaporate-Code+`, achieved an average 110x reduction in LLM processing tokens while outperforming state-of-the-art baselines by 3.2 F1 points, demonstrating high generality across formats and domains.

Key insights

• Large Language Models (LLMs) can enable highly general systems for structured data extraction from heterogeneous data lakes using fixed prompts.
• LLM-powered code synthesis offers a cost-effective alternative to direct LLM inference for large-scale data processing, achieving sublinear cost scaling relative to data size.
• Weak supervision can effectively aggregate noisy, machine-generated functions, even when their individual quality is low, to significantly boost extraction quality.

Summary by alphaXiv.