Reproduce “Resource-Efficient Speech Quality Prediction through Quantization Aware Training and Binary Activation Maps” on OpenResearch

About this paper

Mattias Nilsson, Riccardo Miccini, Clément Laroche, Tobias Piechowiak, Friedemann Zenke

Researchers from Friedrich Miescher Institute for Biomedical Research, University of Basel, GN Audio, and Technical University of Denmark developed a mixed-precision quantization strategy for Speech Quality Prediction (SQP) models, utilizing Binary Activation Maps (BAMs) with Quantization Aware Training (QAT) and 8-bit weights. This approach reduced memory footprint by 25-fold and simplified 171 million convolutional operations while maintaining predictive performance comparable to full-precision models.

Key insights

• Post-training binarization of activations severely degrades model performance, demonstrating the critical need for Quantization Aware Training (QAT) when implementing extreme quantization.
• Binarizing only activations (BAMs) with QAT effectively recovers predictive performance, achieving accuracy levels on par with full-precision models.
• Attempting full binarization (both activations and weights) significantly compromises model accuracy, suggesting a limit to extreme quantization for this specific task and architecture.

Summary by alphaXiv.