Reproduce “Debias-SparseGPT: Bias-Aware Pruning for Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Irina Proskurina, Guillaume Metzler, Antoine Gourru, Julien Velcin
Debias-SparseGPT modifies SparseGPT’s second-order pruning by preserving representational differences between paired demographic texts while selecting and reconstructing weights. For instruction-tuned LLaMA-3.1-8B at 1:4 sparsity, UnQover accuracy is 60.46% versus 35.60% for SparseGPT, with similar MMLU accuracy and perplexity. However, at Qwen-2.5-7B-Instruct’s 2:4 setting with StereoSet-only calibration, UnQover is 24.94% versus 28.45%, and DTO is 0.645 versus 0.616.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.