Reproduce “Neural Architecture Search by Learning a Hierarchical Search Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mehraveh Javan Roshtkhari, Matthew Toews, Marco Pedersoli
A method is introduced that learns a hierarchical search space for Monte-Carlo Tree Search (MCTS) in Neural Architecture Search (NAS) by grouping architectures based on their functional similarity rather than structural components. This approach enables more efficient exploration, achieving 76.7% top-1 accuracy on ImageNet with approximately 7 GPU days of computation while outperforming several established NAS methods requiring more resources.
Key insights
- Replacing a predefined structural hierarchy with a learned functional (semantic) hierarchy in MCTS-based NAS leads to more efficient and effective exploration.
- Architectures with similar functional behavior, rather than structural similarity, should be grouped together to improve the quality of conditional probability approximations in the search process.
- Simultaneously searching for architectures and training the supernet provides a feedback loop that improves both the model and the search guidance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.