Reproduce “WindGP: Efficient Graph Partitioning on Heterogenous Machines” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Li Zeng, Haohan Huang, Binfan Zheng, Kang Yang, Shengcheng Shao, Jinhua Zhou, Jun Xie, Rongqian Zhao and 1 more
WindGP, an edge-centric partitioning algorithm, is explicitly designed for heterogeneous machines, integrating optimizations across memory capacity, computing power, and network bandwidth within a unified framework, resulting in reduced maximum total execution cost for distributed graph algorithms.
Key insights
- Optimally allocating edge capacities per machine during preprocessing, considering heterogeneous compute power and memory, is fundamental for balancing computational load and ensuring memory adherence.
- A best-first graph exploration strategy, utilizing degree-balanced and border-aware expansion, is effective for creating cohesive partitions that reduce inter-machine communication.
- Iterative local search with destroy-and-repair and re-partition operators can refine initial partitions by globally adjusting edge assignments to reduce the maximum total cost across all machines, overcoming local optima.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.