Reproduce “A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Rishi Shah, Rishav Shrestha

Researchers at E3A Healthcare introduced a contract-grade verifier to rigorously assess GPU kernel correctness, revealing that 62.1% of previously accepted LLM-generated kernels contained silent errors, with 39.5% failing tolerance-free checks. The verifier also successfully certified a new native Blackwell training backward for the gated-linear-recurrence (GDN) family, addressing specific hardware constraints.

Key insights

  • A "correctness illusion" in LLM-generated GPU kernels masks a high prevalence of silent errors due to insufficient and overly lenient testing methodologies.
  • Tolerance-free gates are critical for detecting fundamental correctness issues (e.g., non-finite propagation, determinism) that floating-point tolerances cannot sufficiently address.
  • High-performance native kernel development benefits significantly from rigorous, multi-layered verification against analytically derived error bounds and double-precision oracles to ensure stability and accuracy.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.