Reproduce “Hypothesis Generation with Large Language Models” on OpenResearch

About this paper

Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, Chenhao Tan

Researchers at the University of Chicago and TTIC developed HypoGeniC, an iterative, UCB-inspired algorithm that enables large language models to generate high-quality, interpretable hypotheses directly from labeled data. This approach consistently outperforms traditional LLM baselines and sometimes supervised models in classification tasks, recovering known theories and uncovering novel scientific insights that generalize across models and out-of-distribution datasets.

Key insights

• Large language models can be effectively guided to generate and refine high-quality, data-driven hypotheses from empirical examples.
• An iterative, UCB-style reinforcement learning approach enables LLMs to learn from extensive data despite context window limitations by focusing on examples where current hypotheses struggle.
• The generated hypotheses are highly generalizable, being interpretable by humans and transferable across different LLMs and out-of-distribution datasets, indicating the capture of abstract underlying patterns.

Summary by alphaXiv.