Reproduce “Spark: A System for Scientifically Creative Idea Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aishik Sanyal, Samuel Schapiro, Sumuk Shashidhar, Royce Moon, Lav R. Varshney, Dilek Hakkani-Tur
SPARK, a system developed by researchers from Spiral Works, University of Illinois, Urbana-Champaign, and University of Michigan, automates scientific idea generation by combining literature retrieval, LLM-based generation, and a specialized AI peer-reviewer called JUDGE. The system successfully generated and filtered over 10,000 AI research ideas, with the JUDGE model demonstrating enhanced critical evaluation capabilities compared to general LLMs.
Key insights
- Effective scientific idea generation requires coupling LLM-based synthesis with a dedicated, critically-trained evaluation model to overcome general LLMs' agreeable bias.
- Structured datasets derived from peer review data, focusing on conceptual aspects, are vital for training AI evaluators to provide academic-standard critiques.
- Integrating literature retrieval and a multi-agent generation-evaluation loop enables a scalable and grounded approach to computational creativity in science.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.