Reproduce “Model LEGO: Creating Models Like Disassembling and Assembling Building Blocks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiacong Hu, Jing Gao, Jingwen Ye, Yang Gao, Xingen Wang, Zunlei Feng, Mingli Song
Researchers at Zhejiang University, Carnegie Mellon, and NUS developed Model LEGO, a framework for disassembling trained neural networks into task-aware components and assembling them into new models without requiring additional training. This approach facilitates flexible model creation and reuse, demonstrating improved or comparable performance for specific tasks while reducing computational overhead.
Key insights
- Deep neural networks exhibit inherent modularity, akin to biological visual systems, allowing for the identification and extraction of distinct "task-aware components" from trained models.
- A "zero-shot" approach to model disassembly and assembly is feasible, enabling the creation of new, task-specific models from pre-trained components without requiring further training or fine-tuning.
- Balancing component integration with potential interference is critical; while combining components can reduce inference time, excessive complexity in multi-task assembly may degrade performance without fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.