Reproduce “Towards Steering without Sacrifice: Principled Training of Steering Vectors for Prompt-only Interventions” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuntai Bao, Qinfeng Li, Xinyan Yu, Ge Su, Wenqi Zhang, Liu Yan, Haiqin Weng, Jianwei Yin and 1 more
The paper presents Prompt-Only Steering Vectors (PrOSVs), which control concepts in an LLM’s answer by jointly learning a steering factor and direction, then adding the intervention only to selected prompt-token representations during prefill. On AXBENCH, Qwen2.5-32B at layer 32 scores 1.102 with PrOSV versus 0.982 for FSSV under SimPO, across 100 concepts; this comparison does not establish universal superiority.
Key insights
- Neural network scaling theory provides a principled framework for stable and efficient joint training of steering factors and directions, eliminating heuristic hyperparameter tuning.
- Effective concept-based steering does not require full-sequence interventions, as Prompt-Only Steering Vectors (PrOSVs) can achieve control by modifying only input prompt tokens.
- PrOSVs preserve the model's original attention patterns more effectively than full-sequence interventions, contributing to better maintenance of general model utility and generation quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.