Reproduce “Model Whisper: Steering Vectors Unlock Large Language Models' Potential in Test-time” on OpenResearch

About this paper

Xinyue Kang, Diwei Shi, Li Chen

Researchers from Tsinghua University developed Model Whisper, a method using Test-Time Steering Vectors (TTSV) to enhance Large Language Models' performance on specific tasks at inference time. TTSV provides holistic guidance by injecting a continuous signal into the model's embedding space, achieving a 55.95% relative gain on the Qwen2.5-Math-7B model without altering core model parameters.

Key insights

• Directly guiding LLM's internal computational trajectory via continuous steering vectors in the embedding space is more effective than output-level adjustments or discrete prompts.
• A small, optimized set of steering vectors can significantly influence and amplify a linear bias through the deep network, directing the model towards more confident and accurate states.
• Learned steering vectors exhibit robust cross-distribution generalization, indicating they capture fundamental reasoning patterns rather than just specific data biases.

Summary by alphaXiv.