Reproduce “StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jake Hyun, Ziyu Zhao, Jae-sun Seo, Mohamed S. Abdelfattah
StrataFlux keeps quantized weights and the key–value cache in GPU memory while fetching selected precision corrections from slower memory. It selects weight channels and KV token groups likely to matter, overlapping transfers with computation. On RTX 5090, Qwen3-8B’s AIME accuracy rose from 11.0% to 47.1% with YAQA 2-bit weights and 2/2-bit resident KV, at unchanged GPU memory use, in that setting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.