Reproduce “Sharding Prevents LLM Oversight Failures and Adversarial Exploitation” on OpenResearch

About this paper

Victor Akinwande, J. Zico Kolter, Aran Nayebi

Researchers at Carnegie Mellon University introduce sharding as a method to improve Large Language Model (LLM) oversight by distributing decision-making across multiple LLM calls. This architectural intervention enhances evaluation accuracy and robustness against adversarial attacks, demonstrating that an LLM's decision load, not solely its total computational budget, critically impacts its performance as an oversight mechanism.

Key insights

• An LLM's 'decision load' within a single call is a primary bottleneck for oversight accuracy, distinct from and more impactful than simply increasing the total computational budget.
• Overloaded LLM judges are highly susceptible to 'best-of-N presentation attacks,' which exploit their blind spots by generating multiple artifact presentations to maximize the acceptance of genuinely unmet requirements.
• Sharding effectively mitigates load-induced errors and defends against adversarial exploitation by ensuring adequate scrutiny for each decision, while more elaborate within-shard structures offer diminishing returns for basic accuracy.

Summary by alphaXiv.