Reproduce “Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junbo Zhao, Xuefei Ning, Enshu Liu, Binxin Ru, Zixuan Zhou, Tianchen Zhao, Chen Chen, Jiajin Zhang and 2 more
A Dynamic Ensemble Performance Predictor framework was developed to mitigate the "cold-start" problem in predictor-based Neural Architecture Search by dynamically fusing knowledge from multiple low-fidelity information sources. The approach yielded consistent improvements in Kendall's Tau correlation coefficients (e.g., from 0.2549 to 0.7064 with 1% data on NDS-ResNet) and found higher-performing architectures with reduced query budgets across various search spaces.
Key insights
- The utility of various low-fidelity information types for NAS performance prediction is inconsistent, varying significantly across different search spaces and data volume regimes.
- A dynamic ensemble approach, with a gating network, effectively learns to emphasize beneficial low-fidelity experts while de-emphasizing less helpful ones, thereby mitigating the risk of performance degradation.
- Architecture-conditioned dynamic weighting of expert contributions is crucial for robustly leveraging diverse low-fidelity sources, eliminating the need for manual selection of proxies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.