Reproduce “Transferable Graph Optimizers for ML Compilers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yanqi Zhou, Sudip Roy, Amirali Abdolrashidi, Daniel Wong, Peter Ma, Qiumin Xu, Hanxiao Liu, Phitchaya Mangpo Phothilimthana and 4 more

GO coordinates device placement, scheduling, and operation fusion across computation graphs using GraphSAGE representations and segmented recurrent attention, sampling node actions in parallel across iterations. The authors report 33–60% lower step time than TensorFlow defaults and an average 7.8% improvement over optimizing the tasks separately. Scheduling and fusion results use an analytical model; only placement is evaluated on hardware.

Key insights

  • Leveraging Graph Neural Networks alongside Transformer-XL inspired architecture enables efficient and scalable learning of graph representations and optimization decisions for large graphs.
  • Non-autoregressive decision generation drastically improves search efficiency and training time by processing entire graphs simultaneously, while feature modulation enhances generalization to unseen graph structures.
  • Jointly optimizing multiple, interdependent compiler tasks with a shared policy and recurrent attention layers yields synergistic performance improvements that surpass single-task optimizations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.