Reproduce “Separating Common and Unique Directions for Model Merging” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shiting Wang, Yingjie Zou, Ke Li
CUSD separates task updates into broadly shared directions and task-dominated residuals before applying an arithmetic merging operator. It classifies directions by variation in update magnitudes across tasks. In two example layers, the highest-dispersion decile held 53% and 77% of directional energy. Across 44 combinations, it improves 42, with one tie and one small decrease.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.