Reproduce “Breaking the HBM Bit Cost Barrier: Domain-Specific ECC for AI Inference Infrastructure” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rui Xie, Asad Ul Haq, Yunhua Fang, Linsen Ma, Sanchari Sen, Swagath Venkataramani, Liu Liu, Tong Zhang
A domain-specific error-correcting code (ECC) framework enables the use of cheaper, less reliable High-Bandwidth Memory (HBM) for AI inference infrastructure. This framework, developed by researchers at Rensselaer Polytechnic Institute and IBM T.J. Watson Research Center, allows for raw HBM bit error rates (BERs) up to 10^-3 while retaining 78% of ideal throughput and maintaining inference accuracy, significantly reducing HBM bit cost.
Key insights
- The reliability paradigm from storage systems, tolerating high raw bit error rates with powerful system-level ECC, can be effectively applied to High-Bandwidth Memory (HBM) for cost reduction.
- AI inference workloads offer unique characteristics, such as predominantly sequential access patterns and unequal bit criticality in numerical formats, which can be leveraged for highly efficient, domain-specific error correction.
- Moving error correction responsibility from fixed on-die HBM solutions to a flexible host-side memory controller enables the use of stronger codes and workload-aware optimizations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.