Reproduce “Mitigating Selection Bias with Node Pruning and Auxiliary Options” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hyeong Kyu Choi, Weijie Xu, Chi Xue, Stephanie Eckman, Chandan K. Reddy

Researchers at the University of Wisconsin-Madison and Amazon developed Bias Node Pruning and Auxiliary Option Injection to address selection bias in large language models tackling multiple-choice questions. Their methods improved answer accuracy and reduced bias across various LLMs and datasets, with Auxiliary Option Injection also effective in black-box settings.

Key insights

  • Selection bias is an internal model phenomenon, specifically localized to the final layers of the decoder and its interaction with the output projection matrix.
  • The bias is more pronounced in instances where the LLM produces incorrect answers compared to when it provides correct ones.
  • The learned bias vector, computed on one dataset, demonstrates generalizability and can effectively debias models across different datasets, suggesting a common underlying bias mechanism.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.