Reproduce “Accelerating CRONet on AMD Versal AIE-ML Engines” on OpenResearch

About this paper

Kaustubh Mhatre, Vedant Tewari, Aditya Ray, Farhan Khan, Ridwan Olabiyi, Ashif Iquebal, Aman Arora

Researchers implemented the CRONet neural network for topology optimization on AMD Versal AIE-ML engines, achieving up to 2.29x lower inference latency and 3.79x greater energy efficiency compared to an Nvidia T4 GPU. This implementation successfully ran the entire complex, hybrid CNN-RNN model with all intermediate data and weights residing fully on-chip.

Key insights

• BF16 quantization provides equivalent solution accuracy to FP32 for CRONet's topology optimization, enabling efficient deployment without degrading results.
• The AIE-ML's dataflow architecture and extensive on-chip memory hierarchy are highly effective for low-latency, batch-size-one inference, particularly for complex, iterative neural networks like CRONet.
• Deep hardware-software co-design, including specialized kernel development and intelligent resource placement, is essential for successfully deploying and optimizing heterogeneous DNNs on adaptive SoC platforms.

Summary by alphaXiv.