Reproduce “The Universal Weight Subspace Hypothesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Prakhar Kaushik, Shravan Chaudhari, Ankit Vaidya, Rama Chellappa, Alan Yuille
The authors test whether same-architecture models can be represented by shared, low-dimensional weight directions and task-specific coefficients. Across 500 Mistral-7B LoRAs, variation concentrates in 16 or fewer directions. In image classification, a 16-dimensional ViT subspace scores 94.1 ± 2.0 IID and 87.8 ± 1.5 OOD, versus full training’s 94.4 ± 1.7 and 91.3 ± 2.1; universality across architectures remains unestablished.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.