Reproduce “FTTN: Feature-Targeted Testing for Numerical Properties of NVIDIA & AMD Matrix Accelerators” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyi Li, Ang Li, Bo Fang, Katarzyna Swirydowicz, Ignacio Laguna, Ganesh Gopalakrishnan
A feature-targeted testing methodology was developed to characterize undocumented numerical properties of NVIDIA and AMD matrix accelerators. The methodology reveals distinct behaviors in subnormal number support, internal precision, rounding modes, and FMA unit widths, demonstrating that identical matrix operations can produce results varying by six orders of magnitude across different hardware.
Key insights
- Numerical properties of matrix accelerators, including internal precision and rounding, vary significantly across different GPU vendors and generations, often without public documentation.
- These undocumented hardware differences can lead to severe numerical discrepancies, exemplified by a simple matrix multiplication yielding results varying by six orders of magnitude.
- Systematic, feature-targeted testing is crucial for understanding and mitigating the risks of numerical non-portability in high-performance computing and machine learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.