Reproduce “Direct Hidden-State Alignment: Mapping and Controlling Preference Expression in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Fansheng Zhang, Shengran Guo, Zexiao Wang, Liang Yuan, Jiyuan Chen, Ruikun Luo
The paper maps preference-related effects of model components and uses them to guide localized hidden-state control while keeping base weights frozen. On 300 ConFiQA inputs with Llama-3-8B, opposing effects above 0.5 logit-margin units appeared in 283 inputs. After DPO, effects were reorganized, not eliminated. CAST prefill scored lower than DPO on PC but higher on EM, showing task-metric-dependent results.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.