Reproduce “Finer is Better (with the Right Scaling)” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Clemens Schaefer, Gil Tabak
Researchers from Google LLC elucidated why reducing microscaling block sizes sometimes degrades large language model quality with E4M3 scaling factors, demonstrating that finer granularity improves quality when employing a "prevent-zero scale" or "4-over-6" methodology, thereby resolving the observed paradox. These interventions led to substantial reductions in perplexity gap for affected models.
Key insights
- The block-size paradox is not an intrinsic limitation of finer granularity or the FP4 format, but rather an artifact of suboptimal scale selection heuristics.
- Degradation arises from E4M3 scaling factors underflowing to zero or pushing a disproportionate number of elements into coarse, high-magnitude FP4 quantization bins (e.g., between 4.0 and 6.0).
- The "4-over-6 methodology" successfully resolves this by adaptively selecting scale bounds, while the "prevent-zero scale" addresses extreme underflow.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.