Reproduce “Calibration of Large Model Quantization based on HiFloat4 Format” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jing Huang, Yu Cheng, Ziwei Yu, Lan Zhao, Sitan Li, Guangxiu Yao, Xin Wang, Yuanyong Luo and 6 more

This paper adapts calibration to HiF4, a four-bit format with a three-level hierarchy sharing scale metadata across 64-value groups. HiGPTQ processes each group together; HiAutoRound and HiFlatQuant adjust quantization parameters or transformations. Across five models and seven tasks in simulated 4-bit evaluations, all three methods improve on direct HiF4 conversion. HiGPTQ averages 73.708 versus 72.804, within one point of BF16; this is aggregate accuracy evidence, not a hardware-efficiency measurement.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.