Reproduce “Preserving Updates in Low-Bit Recurrent State Quantization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haotian Li, Liran Dong, Kaicheng Xiao, Guoliang Xing
The authors present a training-free method for quantizing recurrent states in four tested 1.3B models. It preserves small updates in a low-rank bypass and corrects some rounding reversals. On WikiText-103 at 8K tokens, RetNet word-level perplexity is 28.80 with the method, versus 1193.21 with INT4 and 24.30 with FP32 states, using 5.25 average bits per state element; it does not match FP32.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.