Reproduce “Value Alignment Tax: Measuring Value Trade-offs in LLM Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiajun Chen, Hua Shen
Researchers at NYU Shanghai introduce the Value Alignment Tax (VAT) framework, a multi-dimensional approach to quantify how alignment interventions reshape a Large Language Model's (LLM) entire value system, not just the target value. The framework reveals that different alignment strategies incur distinct systemic coordination costs, often reflecting human value structures, and identifies potential latent alignment risks through unevenly distributed value shifts.
Key insights
- Alignment interventions achieving comparable on-target gains can incur substantially different System-Level Alignment Tax (nVAT) values, indicating varying systemic coordination costs not detectable by target-only metrics.
- Alignment-induced coordination load is unevenly distributed across values, with specific values (e.g., Conformity, Tradition, Security) consistently emerging as 'coordination hubs' across different steering objectives.
- Different alignment strategies, such as Supervised Fine-tuning (SFT) and Direct Preference Optimization (DPO), traverse distinct regions of the gain–tax space, inducing qualitatively different systemic coordination modes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.