Reproduce “Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yukiya Horiba, Hakaze Cho, Takahiro Yakoh
The method splits task vectors (TVs) into task-recognition (TR) and task-learning (TL) components using candidate-label output directions, while holding a residual fixed. It searches component weights with Bayesian optimization without changing model parameters. On 500 independent SST-2 examples, Llama 3.2 1B accuracy rose from 11.2% to 79.6%, a result specific to that setup and not proof of isolated mechanisms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.