Reproduce “Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Define” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Saurabh Kumar Singh, Yogeshwar Singh Dadwhal, Malhar Vedak
The authors compare rare-word inclusion and definition performance across 27 quantized artifacts from 13 model families, testing 429 frequency-validated English words in models spanning 0.35B–14B parameters. At Q2_K, Granite-4.0-H-1B definition accuracy falls 43.6% relative to FP16, while word inclusion falls 2.1%. Small models often show selective losses, but larger models are mostly robust, with exceptions; findings apply to tested artifacts and settings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.