Reproduce “SISA: A Scale-In Systolic Array for GEMM Acceleration” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Luigi Altamura, Alessio Cicero, Mateo Vázquez Maceiras, Mohammad Ali Maleki, Pedro Trancoso

Researchers at Chalmers University of Technology and University of Gothenburg developed SISA, a scale-in systolic array architecture that partitions processing elements into independently controllable slabs to efficiently accelerate General Matrix-Matrix Multiplication (GEMM) for Large Language Model (LLM) inference. SISA achieved up to 8.52x speedup and 93% Energy-Delay-Product (EDP) reduction for small, skewed GEMMs, while introducing a total chip area overhead of approximately 5.44%.

Key insights

  • Dynamically partitioning a systolic array into independent, power-gated slabs allows for fine-grained resource allocation and significantly improves PE utilization across a wide range of GEMM sizes, particularly for skewed matrices common in LLMs.
  • The 'scale-in' design concept provides both the high computational density of a large monolithic array for optimal workloads and the flexibility to efficiently handle smaller or highly skewed matrix multiplications.
  • Careful design of the memory hierarchy, dataflow, and slab fusion mechanism enables substantial performance and energy efficiency gains with minimal architectural and area overhead.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.