Reproduce “From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search” on OpenResearch

About this paper

Liping Deng, MingQing Xiao

Researchers at the University of California, Riverside and Southern Illinois University Carbondale developed a meta-learning predictor for Neural Architecture Search by reformulating performance prediction as a conditional function inference problem using a Convolutional Neural Process. This approach, which meta-learns from synthetic tasks, demonstrably improves generalization in data-scarce environments and achieves competitive top-K architecture selection on NAS-Bench benchmarks.

Key insights

• Reframing performance prediction as conditional function inference, rather than fixed regression, allows for improved generalization capabilities in data-scarce NAS environments.
• Meta-learning through synthetic task generation and explicit context-target splitting within the ConvNP framework is essential for training predictors that robustly infer performance from limited, variable observations.
• Optimizing a combined objective of regression and ranking losses provides a more effective training signal for the practical goal of selecting high-performing architectures compared to relying solely on global ranking metrics.

Summary by alphaXiv.