Reproduce “Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyan Jiang, Lin Zhang, Jiayi Zhang, Qingsong Yang, Guimin Hu, Di Wang, Lijie Hu
Researchers at MBZUAI and KAUST developed Multi-Subspace Representation Steering (MSRS), a framework that leverages SVD-guided orthogonal and shared subspaces to adaptively align Large Language Models with multiple desired attributes simultaneously. This approach consistently optimized attributes like truthfulness and bias without typical trade-offs, preserving general NLP capabilities with average gains of +3.8% on HellaSwag and +4.9% on GLUE for Llama3-8B-Instruct.
Key insights
- Allocating orthogonal subspaces to each attribute and a shared subspace for common directions effectively mitigates interference while capturing synergistic features.
- Adaptive allocation of subspace sizes, guided by SVD's singular values, ensures that each attribute receives appropriate expressive capacity based on its complexity.
- Dynamic weighting and a multi-component loss function enable precise, disentangled, and integrated control over multiple LLM attributes without degrading general capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.