Reproduce “On Latency Predictors for Neural Architecture Search” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yash Akhauri, Mohamed S. Abdelfattah
NASFLAT is a few-shot latency-prediction framework for hardware-aware neural architecture search that combines operation-specific hardware embeddings, diverse architecture sampling, graph neural networks, and auxiliary architecture encodings. Across NASBench-201 and FBNet transfer tasks, it outperformed HELP and MultiPredict on 11 of 12 device splits, improving ranking quality by an average of 22.5% and reducing predictor-related NAS time by roughly 5.8×.
Key insights
- Hardware diversity matters more than simply increasing the number of source-device samples; adding data from highly similar devices can encourage overfitting and hurt transfer.
- Architecture-encoding-based sampling improves few-shot calibration over random selection in most evaluated device pools, with cosine-based selection generally more reliable than KMeans.
- Treating hardware as an operation-dependent context is more effective than supplying only a global device representation, reflecting differences in operator implementations, fusion, pipelining, and memory behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.