Reproduce “Rerouting LLM Routers” on OpenResearch

About this paper

Avital Shafran, Roei Schuster, Thomas Ristenpart, Vitaly Shmatikov

LLM routers that choose between models based on query difficulty can be manipulated by short, reusable token sequences appended to user inputs. Across four open-source routers, these “confounder gadgets” redirected roughly 73–100% of queries from weaker to stronger models with little response-quality degradation; tests on commercial services also produced cost increases ranging from about 1.7× to over 8×, while perplexity-based defenses weakened against optimized, natural-looking gadgets.

Key insights

• Routing is a distinct security control plane: an input can preserve the requested task and largely preserve answer quality while still changing the model, cost, and inference path.
• Reusable gadgets can be generated once and applied across many unrelated queries; transfer attacks remain effective even when the attacker does not know the target router.
• Perplexity filtering detects basic unnatural gadgets well, but jointly optimizing for low perplexity substantially reduces detectability while retaining rerouting ability.

Summary by alphaXiv.