Reproduce “Compile by Training: Turning Natural-Language Specifications into Local Neural Functions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuntian Deng, Pengyu Nie, Stuart Shieber
The research introduces "Compile by Training," a method for transforming natural-language function specifications into local, reusable neural functions. This approach achieved an 83.6% LLM Exact Match (LEM) score on the challenging FuzzyBench-Hard benchmark, representing a 61.2 percentage point improvement over previous fast compilation techniques by leveraging additional computational investment during the one-time compilation phase.
Key insights
- Additional computational investment during a one-time compilation phase, rather than fast one-shot prediction, significantly enhances the semantic correctness of neural functions.
- An interactive, minute-scale compilation process can be made practical by overlapping teacher synthesis with model training and implementing efficient job coordination and caching.
- Large language models can be effectively leveraged as 'tool builders' for generating specialized, local neural functions, reducing continuous runtime dependencies and fostering reusable AI components.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.