Reproduce “Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Lucas Sheneman

A differentiable meta-circular interpreter (DMCI) is introduced, compiling a substantial, self-hosting subset of Scheme into computation graphs compatible with standard automatic differentiation. This system allows complex, executable programs to be directly optimized with gradient descent, offering a "compile once, differentiate everywhere" approach for scientific models.

Key insights

  • By compiling and differentiating the interpreter itself, any program evaluated by it automatically acquires differentiability without needing individual recompilation or custom gradient derivations, enabling a "compile once, differentiate everywhere" paradigm.
  • The use of fixed-width tagged-value tensors for all runtime values, combined with a write-once dictionary-backed heap, allows gradients to traverse discrete control flow and memory allocations within the interpreter transparently.
  • The system provides a robust framework for hybrid AI workflows, allowing large language models to propose discrete program structures while DMCI provides exact gradients for efficient continuous parameter calibration.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.