Reproduce “AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities” on OpenResearch

About this paper

Sandy Abdo, Bill Kapralos, Priyamvada Tripathi, KC Collins, Adam Dubrowski

A narrative review surveys AI-based generative models for sound effect synthesis over the last five years, categorizing them by input modality to analyze their impact on audio quality, controllability, and contextual relevance. It highlights the prevalence of diffusion-based architectures and identifies key challenges in temporal synchronization and objective-subjective evaluation alignment.

Key insights

• Diffusion-based architectures, particularly latent diffusion models, have become prominent for their ability to achieve high-fidelity and computationally efficient sound effect generation.
• The integration of large language models and retrieval mechanisms enhances semantic understanding and improves the generation of contextually relevant and specific sound events.
• Significant challenges persist, including precise temporal synchronization, aligning objective metrics with subjective human perception, and balancing user control with generative diversity.

Summary by alphaXiv.