Reproduce “Large Language Model Data Generation for Enhanced Intent Recognition in German Speech” on OpenResearch

About this paper

Theresa Pekarek Rosin, Burak Can Kaplan, Stefan Wermter

Researchers at the University of Hamburg developed an enhanced German intent recognition system for elderly speakers by combining a domain-adapted Automatic Speech Recognition (ASR) model with synthetic text data generated by Large Language Models (LLMs). The method achieved 83.01% accuracy on a real-world dataset when using a Whisper+DistilBERT model pre-trained on data from the German-specific LeoLM, demonstrating improved generalization over approaches relying solely on limited real data.

Key insights

• Smaller, domain-specific LLMs like LeoLM can generate higher-quality synthetic data for specific language tasks, leading to better generalization than larger, general-purpose LLMs such as ChatGPT.
• Augmenting real-world datasets with LLM-generated synthetic data substantially enhances the robustness and generalization of intent recognition models against unseen sentences and diverse linguistic patterns.
• DistilBERT models, when pre-trained on LLM-generated data, exhibited strong performance comparable to BERT, while Electra-based models were less effective for this intent recognition task.

Summary by alphaXiv.