Reproduce “TASO: Time and Space Optimization for Memory-Constrained DNN Inference” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuan Wen, Andrew Anderson, Valentin Radu, Michael F.P. O'Boyle, David Gregg
The TASO framework from Trinity College Dublin and the University of Edinburgh introduces an Integer Linear Programming approach to optimize convolutional neural network inference on memory-constrained embedded devices. This method holistically selects optimal primitives and data layouts, effectively balancing execution time and memory footprint. It achieves up to an 8x speedup over greedy algorithms under tight memory budgets and enables significant memory reductions, such as a 2.2x decrease, with minor performance impacts on architectures like GoogleNet.
Key insights
- Optimal trade-offs between execution time and memory footprint can be effectively managed through a holistic ILP approach, providing globally optimal solutions superior to single-objective or greedy methods.
- Generating Pareto frontiers allows system designers to precisely select configurations that balance speed and memory, enabling substantial memory savings (e.g., 2.2x) with minimal performance impacts.
- A workspace-based optimization strategy addresses devices with severely limited RAM by minimizing the peak memory required by any single layer, making layer-by-layer inference feasible for large networks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.