Reproduce “Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aly Kassem, Thomas Jiralerspong, Negar Rostamzadeh, Golnoosh Farnadi
Delta-Crosscoder, a modified Crosscoder approach, reliably identifies internal representation changes induced by narrow fine-tuning in large language models by explicitly modeling activation differences and utilizing task-agnostic contrastive data. The method successfully recovered causally relevant latent features for fine-tuned behaviors across 10 diverse model organisms, outperforming existing SAE-based baselines and producing interpretable artifacts more efficiently than some non-SAE methods.
Key insights
- Explicitly optimizing for activation differences (Delta Loss) directly targets and amplifies the subtle representation changes characteristic of narrow fine-tuning, overcoming the bias of standard reconstruction objectives.
- Constructing contrastive text pairs from task-agnostic data effectively induces and isolates asymmetric behavioral shifts, concentrating activation differences in causally relevant regions.
- Strategic partitioning and masking of latent features (Dual-K sparsity) are essential for dedicating model capacity to fine-tuning-specific changes, preventing them from being overshadowed by more common shared features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.