Reproduce “BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dayou Du, Woon Yee Ng, Guangming Sheng, Nick Lau, Jianyi Cheng, Luo Mai
BitDecoding-FP4 stores most attention history in four-bit microscaling values, corrects numerical errors, and organizes GPU kernels for native FP4 execution. It uses residual query quantization, NVFP4 probability scaling, and key smoothing and rotation. On Qwen3-4B-Thinking-2507, NVFP4 averages 67.27% across seven benchmarks versus 67.69% with BF16. On NVIDIA B300 workloads, peak per-GPU throughput is 2.4 to 3.0 times higher than BF16; this is not a universal guarantee.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.