Reproduce “YaPO: Learnable Sparse Activation Steering Vectors for Domain Adaptation” on OpenResearch

About this paper

Abdelaziz Bounhar, Rania Hossam Elmohamady Elbadry, Hadi Abdine, Preslav Nakov, Michalis Vazirgiannis, Guokan Shang

YaPO introduces a method for fine-grained LLM control by learning sparse activation steering vectors in a pre-trained Sparse Autoencoder's latent space, utilizing a preference optimization objective. The approach demonstrates faster convergence and superior performance in cultural alignment tasks, reducing the implicit–explicit localization gap while preserving the model's general knowledge.

Key insights

• Operating in the disentangled sparse latent space of SAEs yields cleaner gradients, leading to substantially faster and more stable optimization for LLM control.
• Combining preference optimization with sparse representations enables robust, fine-grained control over specific behaviors without degrading the LLM's broader knowledge base.
• The method enhances interpretability by intervening on approximately monosemantic features, offering insights into which conceptual factors are modulated within the LLM.

Summary by alphaXiv.