Reproduce “LinQuant: Functional Invariances for Low-Precision Linear Recurrences” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Neehal Tumma, Philipp Nazari, Rohan Cherukuri, Daniela Rus, T. Konstantin Rusch
LinQuant changes how linear recurrences are represented before quantization while preserving their full-precision computation under stated conditions. It uses rotations, diagonal rescalings, and redistribution of decay or write factors across operands; benefits depend on recurrence structure and quantization scales. On Mamba-2 1.3B WikiText-2, a W4A8/INT8 configuration has perplexity 10.860 versus 10.928 for SSDi8, but this comparison does not isolate rotation’s effect.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.