Reproduce “Cyclotron: Compilation of Recurrences to Distributed and Systolic Architectures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shiv Sundram, Akhilesh Balasingam, Nathan Zhang, Kunle Olukotun, Fredrik Kjolstad
Cyclotron is a compiler framework from Stanford University that translates high-level recurrence equations for streaming dataflow algorithms into efficient, executable programs for both distributed CPU clusters and simulated systolic architectures. The system generated distributed matrix multiplication code that generally outperformed ScaLAPACK and demonstrated comparable performance for triangular solve, while also enabling simulation and design space exploration for complex algorithms like FlashAttention-2.
Key insights
- Recurrence equations, when combined with explicit scheduling directives, offer a powerful and practical declarative model for programming a wide range of streaming dataflow algorithms with dependencies on spatial architectures.
- The "dataflow recurrences" IR effectively unifies the representation of computation and fine-grained communication (sends/receives) in an architecture-agnostic manner, enabling portability and systematic transformation.
- The framework facilitates systematic hardware-software co-design and design space exploration by allowing users to explore different dataflow patterns and evaluate their impact on performance through simulation and real-world distributed execution.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.