Reproduce “Sound2Hap: Learning Audio-to-Vibrotactile Haptic Generation from Human Ratings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yinan Li, Hasti Seifi
A data-driven model, Sound2Hap, developed by researchers at Arizona State University, generates perceptually aligned vibrotactile feedback from diverse environmental sounds by learning from human ratings. This approach significantly outperforms traditional signal-processing methods, achieving higher user ratings for audio-vibration match and improving perceived realism and harmony in haptic experiences.
Key insights
- Human perception of vibrotactile feedback generated by existing signal-processing algorithms varies widely across different environmental sounds, highlighting the need for data-driven, perceptually informed approaches.
- Incorporating human perceptual ratings in a learning-based model significantly enhances the congruence and quality of generated audio-to-vibration mappings compared to rule-based methods.
- The use of a pre-trained audio encoder combined with a custom SEANet-based decoder effectively translates audio latent representations into perceptually meaningful vibrotactile signals with low latency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.