Reproduce “Kilobyte Models: Neural Networks as a Seed and a Quantized Latent” on OpenResearch

About this paper

Sahil Rajesh Dhayalkar

Kilobyte Models introduce a neural network compression technique that stores an integer seed and a quantized latent vector, allowing on-demand regeneration of network weights. This approach achieves extreme storage reductions, such as a 26x reduction for CNNs and a 370x reduction for LSTMs, while maintaining accuracy comparable to full models and enabling highly efficient fine-tuning adapters.

Key insights

• Neural network weights can be regenerated from a very compact, quantized latent vector and a deterministic, seed-generated random basis, achieving extreme compression ratios.
• Quantization-aware training (QAT) applied to the low-dimensional latent vector is crucial for maintaining high accuracy at aggressively low bit widths (e.g., 4-bit), unlike direct post-training quantization of full models.
• The method enables highly storage-efficient parameter-efficient fine-tuning (PEFT), with kilobyte-sized adapters outperforming existing PEFT methods under strict storage budgets.

Summary by alphaXiv.