Reproduce “DFModel: Design Space Optimization of Large-Scale Systems Exploiting Dataflow Mappings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sho Ko, Nathan Zhang, Olivia Hsu, Ardavan Pedram, Kunle Olukotun
Stanford University researchers developed DFModel, a two-layered optimization framework that models and maps general computational workloads onto heterogeneous distributed systems by exploiting dataflow architectures. It achieved up to 6.13x speedup for LLM training over non-dataflow methods and consistently predicted performance with an average 10% upper bound compared to real-world measurements across various AI and HPC tasks.
Key insights
- Dataflow architectures, when combined with optimized dataflow mappings, consistently yield substantial improvements in performance, cost, and power efficiency for modern AI workloads.
- Holistic optimization across the entire memory and interconnection network hierarchy is essential to balance compute, memory, and network latencies and prevent system bottlenecks.
- Accurately identifying optimal parallelization strategies (Tensor, Pipeline, Data Parallelism) and efficient intra-chip kernel management through fusion, tiling, and pipelining is critical for maximizing system performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.