Reproduce “FTRANS: Energy-Efficient Acceleration of Transformers using FPGA” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bingbing Li, Santosh Pandey, Haowen Fang, Yanjun Lyv, Ji Li, Jieyang Chen, Mimi Xie, Lipeng Wan and 2 more
FTRANS presents a framework combining enhanced Block-Circulant Matrix compression with a custom FPGA acceleration design to enable energy-efficient deployment of large Transformer models. The system achieved up to 16x model size reduction with minor accuracy impact and demonstrated up to 27x higher throughput and 81x greater energy efficiency compared to CPU for RoBERTa inference.
Key insights
- A holistic hardware-software co-design approach, integrating algorithmic compression with architectural optimization, is crucial for efficiently accelerating complex Transformer models on FPGAs.
- Enhanced Block-Circulant Matrix (BCM) compression, coupled with 16-bit quantization, effectively reduces Transformer model size by up to 16 times while largely preserving accuracy for standard models and offering a manageable trade-off for large pre-trained models like RoBERTa.
- Well-designed FPGA accelerators can significantly outperform CPUs and high-end GPUs in both throughput and energy efficiency for memory-intensive Transformer inference workloads.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.