Reproduce “Robustifying and Boosting Training-Free Neural Architecture Search” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhenfeng He, Yao Shu, Zhongxiang Dai, Bryan Kian Hsiang Low

RoBoT, an algorithm developed by researchers from the National University of Singapore and MIT, combines existing training-free Neural Architecture Search (NAS) metrics through Bayesian Optimization to create a robust performance estimator. This method then applies a greedy search on top-ranked architectures to bridge the estimation gap, consistently matching or exceeding the performance of the best individual training-free metrics across various NAS benchmarks while reducing computational costs.

Key insights

  • A weighted linear combination of training-free metrics significantly enhances correlation with true performance and provides more consistent results than individual metrics across diverse tasks.
  • Bayesian Optimization effectively finds optimal metric weights by balancing exploration and exploitation, even with limited observations of true architecture performance.
  • A distinct "estimation gap" between training-free scores and actual performance can be systematically quantified and reduced through targeted exploitation of top-ranked architectures.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.