Reproduce “Hyperdimensional Computing vs. Neural Networks: Comparing Architecture and Learning Process” on OpenResearch

About this paper

Dongning Ma, Xun Jiao

Researchers at Villanova University compared Hyperdimensional Computing (HDC) with neural networks (NNs), showing that a canonical HDC model can be conceptually derived from a compact two-layer NN. This NN-derived HDC achieved superior classification accuracy, improving performance by up to 6% on MNIST and 21% on CIFAR-10 compared to conventional HDC models.

Key insights

• A canonical HDC model can be conceptualized as a compact two-layer neural network, with item memory acting as input layer weights and associative memory as classifier layer weights.
• HDC's simplified learning mechanism, characterized by fixed item memory, 'coerce' back-propagation, and a high learning rate, prioritizes efficiency but limits its learning capability compared to gradient-descent NNs.
• The holographic nature of HDC means elements within a class Hypervector are not individually trainable, a fundamental difference from individual neuron weights in NNs.

Summary by alphaXiv.