Reproduce “How Language Models Process Negation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhejian Zhou, Tianyi Zhou, Robin Jia, Jonathan May
The authors investigate how language models process overt negation. They compare positive and negative prompts and use logit tests and interventions to examine competing construction and suppression mechanisms. For Llama-3.1-8B, negative accuracy rises from 50.5% to 67.8% after the best tested Attention Sink intervention. Their evidence favors construction, but does not establish a general fix or sole cause of errors.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.