Reproduce “Surprisingly Strong Performance Prediction with Neural Graph Features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gabriela Kadlecová, Jovita Lukasik, Martin Pilát, Petra Vidnerová, Mahmoud Safari, Roman Neruda, Frank Hutter
A study by researchers including those from Charles University and the University of Freiburg introduces Neural Graph Features (GRAF), a set of interpretable architectural properties, significantly improving performance prediction in Neural Architecture Search. GRAF, particularly when combined with existing zero-cost proxies, achieved superior Kendall tau correlation across various benchmarks and tasks, while also providing insights into task-specific architectural drivers.
Key insights
- Many zero-cost proxies (ZCPs) show direct dependencies on basic architectural properties like convolution counts, limiting their ability to truly correlate with performance or distinguish structurally similar networks.
- Neural Graph Features (GRAF) capture architectural information complementary to ZCPs, demonstrating a synergistic effect that results in substantially improved and more robust performance prediction when combined.
- SHAP analysis of feature importance revealed that optimal architectural properties are highly task-specific, providing interpretable insights into which network design elements are crucial for different objectives (e.g., autoencoding vs. classification).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.