Reproduce “CLOSE: Curriculum Learning On the Sharing Extent Towards Better One-shot NAS” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixuan Zhou, Xuefei Ning, Yi Cai, Jiashu Han, Yiping Deng, Yuhan Dong, Huazhong Yang, Yu Wang
The research introduces a curriculum learning strategy, CLOSE, for one-shot Neural Architecture Search (NAS) supernets, enabled by a novel CLOSENet architecture that dynamically adjusts parameter sharing. This approach improves the correlation between supernet performance estimations and stand-alone architecture performance, achieving superior ranking quality and discovering high-performing architectures without significantly increasing training costs.
Key insights
- A larger parameter sharing extent in supernets facilitates faster initial training and better early-stage ranking, while a smaller extent enables higher saturating ranking quality after extensive training.
- Dynamically adjusting the parameter sharing extent during supernet training, akin to curriculum learning, can effectively combine the benefits of both rapid initial convergence and high final ranking accuracy.
- Decoupling parameters from specific operations and using a learnable, context-aware assignment mechanism allows for a more flexible and adaptable supernet design compared to fixed positional sharing schemes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.