Reproduce “Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Charles Eckert, Xiaowei Wang, Jingcheng Wang, Arun Subramaniyan, Ravi Iyer, Dennis Sylvester, David Blaauw, Reetuparna Das

Researchers from the University of Michigan and Intel Corporation developed "Neural Cache," an approach that transforms existing on-chip cache memory into a computational engine for Deep Neural Networks. This architecture enables bit-serial arithmetic operations directly within SRAM arrays, achieving 18.3x lower inference latency than a CPU and 7.7x lower latency than a GPU for the Inception v3 model, alongside significant energy efficiency improvements.

Key insights

  • It is possible to perform in-situ arithmetic operations (addition, multiplication) directly within standard SRAM cache arrays with minimal circuit modifications.
  • Bit-serial processing enables massive parallelism across the cache structure and offers flexibility in data precision by allowing operations to stop at desired bit positions.
  • Repurposing existing on-chip cache for computation provides a cost-effective DNN acceleration without requiring specialized, dedicated accelerator hardware, with an estimated area overhead of less than 2% of total processor die area.
  • The design can maintain the primary function of cache memory while adding computational capabilities by leveraging existing bitline and wordline structures in SRAM cells.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.