Reproduce “Large Language Models in Resolving Contextual Knowledge Conflicts” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinye Yang, Zhenyang Liu, Ruisi Li, Yuanyuan Lei
The authors introduce ContextConflict, a dataset of 5,781 instances spanning six types of conflict among supplied sources. They test activation steering toward average single-source activations. On Llama-3.1-8B-Instruct Temporal reasoning, accuracy rises from 19.5% to 38.7%, while Faithfulness falls from 53.7% to 49.0%. These results are from tested open-weight models and require internal activation access, adding inference-time overhead.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.