Reproduce “Accelerating ViT Inference on FPGA through Static and Dynamic Pruning” on OpenResearch

About this paper

Dhruv Parikh, Shouyi Li, Bingyi Zhang, Rajgopal Kannan, Carl Busart, Viktor Prasanna

An algorithm-hardware co-design accelerates Vision Transformer (ViT) inference on FPGAs by simultaneously applying static weight and dynamic token pruning. This approach achieved up to a 12.8x latency reduction compared to a high-end CPU and a 3.2x reduction against an NVIDIA GPU, reducing computational complexity by 3.42x with minimal accuracy loss.

Key insights

• Simultaneously applying both static block weight pruning and dynamic token pruning yields greater computational reduction for Vision Transformers than either method alone.
• An algorithm-hardware co-design is essential for efficiently mapping highly irregular, dynamically changing sparse computations onto FPGAs, optimizing custom data paths and on-chip memory.
• Knowledge distillation, coupled with a progressive sparsity scheduler during training, effectively preserves model accuracy when aggressive, combined pruning strategies are applied.

Summary by alphaXiv.