Reproduce “Exploring Efficient-tuning Methods in Self-supervised Speech Models” on OpenResearch

About this paper

Zih-Ching Chen, Chin-Lun Fu, Chih-Ying Liu, Shang-Wen Li, Hung-yi Lee

A comprehensive study demonstrates that efficient tuning methods enable self-supervised speech models to achieve performance comparable to full fine-tuning while reducing trainable parameters by over 90% across diverse speech tasks. These methods also exhibit superior performance and stability in low-resource adaptation scenarios.

Key insights

• Efficient tuning methods can effectively adapt large self-supervised speech models, achieving performance parity with drastic parameter reduction (over 90%) and often outperforming full fine-tuning, especially in low-resource settings.
• Minimal, targeted adjustments to the pre-trained model are crucial for optimal task adaptation, as most efficient methods significantly outperform merely training a prediction head on frozen features.
• The effectiveness of specific efficient tuning methods varies between speech and NLP, with Houlsby adapters and Weighted-sum showing particular strength in speech, while LoRA and Prefix Tuning perform less effectively in this domain.

Summary by alphaXiv.