Reproduce “TinySQL: A Progressive Text-to-SQL Dataset for Mechanistic Interpretability Research” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Abir Harrasse, Philip Quirke, Clement Neo, Dhruv Nathawani, Luke Marks, Amir Abdullah

Researchers from Martian and Apart Research developed TinySQL, a progressively complex text-to-SQL dataset specifically engineered for mechanistic interpretability studies. Their work employs this dataset to train language models and analyze the specific computational circuits and layer-wise processes involved in generating SQL queries, providing a controlled environment for understanding model internals.

Key insights

  • Language models generate SQL queries in a consistent two-phase process: initially recognizing the high-level structural intent in early layers, followed by resolving schema-specific elements in deeper layers.
  • Edge Attribution Patching (EAP) effectively identifies precise attention heads in smaller models but encounters scalability challenges in isolating granular circuits in larger models, often attributing importance to MLPs.
  • Sparse Autoencoders (SAEs) successfully learn task-relevant features, and their identified circuits show a high degree of overlap with EAP results, providing a measure of identifiability and supporting their causal validity.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.