Reproduce “N-TORC: Native Tensor Optimizer for Real-time Constraints” on OpenResearch

About this paper

Suyash Vardhan Singh, Iftakhar Ahmad, David Andrews, Miaoqing Huang, Austin R. J. Downey, Jason D. Bakos

N-TORC presents a two-phase framework that optimizes machine learning model deployment on FPGAs, specifically addressing the challenges of achieving real-time latency and minimizing hardware resource costs. It integrates neural architecture search with a Mixed Integer Program solver, accelerating hardware optimization up to 1000 times compared to stochastic methods while satisfying strict latency requirements.

Key insights

• Specialized, data-driven performance models for HLS4ML layers provide significantly higher prediction accuracy (R² > 0.99 for Conv1D/Dense, >0.93 for LSTM) compared to general HLS prediction methods.
• The HLS4ML 'reuse factor' offers fine-grained control over hardware parallelism, enabling precise trade-offs between latency and resource consumption at the layer level.
• Combining a high-level network architecture search with a low-level, MIP-based hardware optimization effectively navigates the complex design space for real-time FPGA deployment.

Summary by alphaXiv.