Reproduce “Analyzing and Fine-Tuning Whisper Models for Multilingual Pilot Speech Transcription in the Cockpit” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kartheek Kumar Reddy Nareddy, Sarah Ternus, Julia Niebling
Researchers at the German Aerospace Center (DLR) adapted Whisper models for multilingual pilot speech in the cockpit, reducing the Word Error Rate (WER) from 68.49% to 26.26% by combining parameter-efficient fine-tuning with custom normalization schemes. This tailored approach addresses the unique challenges of high noise and specific aviation jargon in cockpit environments.
Key insights
- Pre-trained Whisper models exhibit high Word Error Rates (WERs) on cockpit speech, necessitating domain adaptation.
- Custom normalization schemes, specifically designed for pilot communication characteristics (e.g., ICAO alphabet, filler words), substantially reduce WER even before fine-tuning.
- The combination of LoRA fine-tuning and proposed normalization strategies yields a synergistic effect, achieving the most significant performance improvements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.