Reproduce “Embedded Binarized Neural Networks” on OpenResearch

About this paper

Bradley McDanel, Surat Teerapittayanon, H.T. Kung

Harvard University researchers developed Embedded Binarized Neural Networks (eBNNs) to enable deep learning inference on microcontrollers with severe memory constraints. eBNNs achieve a 32x reduction in temporary memory usage by processing intermediate results within 'fused binary blocks,' demonstrating high accuracy (e.g., 97.86% on MNIST) and significant energy savings on devices with only tens of kilobytes of memory.

Key insights

• Intermediate floating-point results do not need to be stored in large matrices between layers; they can be processed and binarized immediately within fused computational blocks.
• Optimizing for memory efficiency by processing data in a streaming, sequential fashion is crucial for embedded devices, differing from GPU-optimized parallelism.
• A deliberate tradeoff of additional computation (recomputing values) for memory savings is a beneficial strategy for severely memory-constrained systems.

Summary by alphaXiv.