Reproduce “SonicGauss: Position-Aware Physical Sound Synthesis for 3D Gaussian Representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chunshi Wang, Hongxing Li, Yawei Luo
Researchers at Zhejiang University developed SonicGauss, a framework to synthesize position-aware impact sounds directly from 3D Gaussian Splatting representations. It employs a multi-stage diffusion model pipeline to generate realistic audio that varies with impact location and material properties.
Key insights
- 3D Gaussian Splatting representations inherently encode sufficient geometric and appearance cues to infer material properties relevant for sound synthesis.
- A multi-stage training approach, incorporating contrastive learning and fine-tuning with real-world data, effectively bridges 3D visual representations with complex acoustic outputs.
- Integrating position encoding via cross-attention allows for generating acoustically distinct sounds based on the precise impact location on an object, capturing nuanced material-position interactions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.