Reproduce “Bitwise Neural Networks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Minje Kim, Paris Smaragdis

Minje Kim and Paris Smaragdis introduce Bitwise Neural Networks (BNNs), which operate entirely on binary values (±1) during inference, replacing traditional arithmetic with bitwise XNOR and bit counting operations. This approach achieves competitive classification accuracy on the MNIST dataset, with only a marginal increase in error (e.g., 0.04% to 0.16% absolute difference) compared to real-valued networks.

Key insights

  • Functional deep neural networks can be constructed to operate entirely on single binary bits for all parameters and signals during inference.
  • Replacing complex arithmetic with elementary bitwise operations (XNOR, bit counting) drastically reduces hardware footprint, memory bandwidth, and power consumption.
  • A two-stage training strategy, using real-valued proxies with compression and noisy backpropagation with binarized parameters, successfully guides the network to an efficient binary configuration.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.