Reproduce “NEST: Network- and Memory-Aware Device Placement For Distributed Deep Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Irene Wang, Vishnu Varma Venkata, Arvind Krishnamurthy, Divya Mahajan

NEST plans how to partition and place model-training workloads by jointly accounting for compute, memory feasibility, and communication costs across network links. Its search combines local parallelism with global placement decisions and represents network locality through communication levels. In a modeled TPUv4-like Fat-Tree evaluation, it reports average throughput 2.43 times Alpa-E’s; these large-cluster results are estimates, not physical training measurements.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.