Reproduce “Preserving Updates in Low-Bit Recurrent State Quantization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haotian Li, Liran Dong, Kaicheng Xiao, Guoliang Xing

The authors present a training-free method for quantizing recurrent states in four tested 1.3B models. It preserves small updates in a low-rank bypass and corrects some rounding reversals. On WikiText-103 at 8K tokens, RetNet word-level perplexity is 28.80 with the method, versus 1193.21 with INT4 and 24.30 with FP32 states, using 5.25 average bits per state element; it does not match FP32.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.