Reproduce “Breaking Information Islands in Sparse Tuning via Small-World Connectivity” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xianchao Guan, Zijun Xiong, Yifeng Wang, Yubo Cui, Yaowei Wang, Guo-Sen Xie, Xin Li, Zheng Zhang

SWIFT adapts pretrained models with sparse trainable updates that mix features within blocks, then use a fixed permutation to route outputs across feature groups; pretrained weights remain frozen, and routing adds no trainable parameters. With the same 0.88% trainable-parameter budget on LLaMA3-8B, SWIFT averages 88.05% across eight commonsense tasks versus DiaBlo’s 87.00%, supporting routing in this setting, not a universal advantage.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.