Reproduce “Bit-Exact AI Inference Verification Without Performance Tradeoffs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Naci Cankaya
This work demonstrates that bit-exact verification of AI inference on NVIDIA GPUs is achievable without performance compromise. It identifies the specific conditions necessary for output reproducibility and introduces a software emulator that precisely models GPU arithmetic, enabling robust auditing of AI systems.
Key insights
- Most perceived "non-determinism" in GPU floating-point computations for AI inference is actually deterministic non-invariance, sensitive to execution path and reproducible if sufficient information is known.
- True non-determinism is rare in modern inference engines, primarily limited to floating-point `atomicAdd` operations within specific integer de-quantization kernels.
- Rounding errors, typically viewed as noise, can be transformed into an auditable "fingerprint" of the exact hardware and software stack, enabling the detection of subtle, numerically relevant modifications.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.