Reproduce “Thinking While Listening: Simple Test Time Scaling For Audio Classification” on OpenResearch

About this paper

Prateek Verma, Mert Pilanci

The Stanford University team introduced "thinking-while-listening," a method for audio classification that generates sequential "reasoning traces" from audio patch predictions. By feeding these traces to a lightly adapted language model, performance matches fully fine-tuned models (e.g., 88.5% Top-1 accuracy on ESC-50) without fine-tuning the heavy audio backbone, effectively leveraging inference-time scaling from LLMs.

Key insights

• Increasing the length of the "sampling trace" at test time consistently improves audio classification performance, validating the "thinking longer" concept in the audio domain.
• A lightweight adaptation of an LLM, specifically by retraining only its embedding matrix, is substantially more effective as a reasoning model compared to complex zero-shot prompting of much larger LLMs.
• It is possible to achieve performance comparable to fully fine-tuned audio models by keeping the heavy audio backbone frozen and employing a "thinking-while-listening" inference strategy.

Summary by alphaXiv.