Reproduce “XtraMAC: An Efficient MAC Architecture for Mixed-Precision LLM Inference on FPGA” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Feng Yu, Hongshi Tan, Yao Chen, Weng-Fai Wong, Bingsheng He

XtraMAC presents a Multiply–Accumulate (MAC) architecture for Field-Programmable Gate Arrays (FPGAs) that efficiently supports mixed-precision and runtime datatype switching in Large Language Model (LLM) inference, improving DSP utilization and compute density over existing solutions. The design achieved 1.2x lower latency and 1.9x higher energy efficiency than an NVIDIA H100 GPU for bandwidth-bound GEMV workloads on FPGAs.

Key insights

  • All integer, floating-point, and mixed-precision multiplications can be unified as an integer mantissa product, allowing a single DSP multiplier datapath for diverse formats.
  • Decoupling integer and floating-point adders, rather than using a complex unified adder, prevents resource waste and latency overheads due to their fundamentally different arithmetic behaviors.
  • Fine-grained bit-level control and dynamic packing of multiple low-precision MACs into disjoint bit regions of a single DSP slice significantly improves hardware utilization and parallelism.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.