Reproduce “Quantization Effects on Bangla Language Understanding in Large Language Models: A Systematic Evaluation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ismail Hossain, Nafi Ullah Shafin, Mohammad Abdullah Al Mumin
Researchers from Shahjalal University of Science and Technology systematically evaluated the effects of post-training quantization on Large Language Models for Bangla language understanding. Their findings indicate that GPTQ-based quantization (GPTQ-Int8, GPTQ-Q8) largely preserved accuracy for Qwen and LLaMA models, while GGUF-W8A16 quantization on GPT-OSS resulted in significant accuracy degradation on reasoning-heavy tasks.
Key insights
- LLM architecture and the specific post-training quantization method significantly influence compression robustness, more so than just parameter count.
- Reasoning-heavy tasks are consistently more susceptible to degradation from quantization than reading comprehension tasks, a pattern observed across both English and Bangla evaluations.
- Bangla's linguistic characteristics may interact with quantization, potentially amplifying performance loss for models not optimally calibrated for such languages, as hypothesized for the GPT-OSS family.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.