Reproduce “Fully Differentiable Neural Forced Alignment via Soft Dynamic Programming” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rotem Rousso, Eyal Cohen, Joseph Keshet
Researchers at The Hebrew University of Jerusalem developed FALCON, a fully differentiable neural architecture for phoneme-level forced alignment, leveraging soft dynamic programming. This system achieved high precision in phoneme and word alignment, outperforming traditional and neural ASR-based methods and demonstrating strong generalization to unseen languages like Dutch and German without additional training.
Key insights
- The integration of a fully differentiable Soft Dynamic Programming (Soft-DP) module is critical for achieving high phoneme boundary estimation accuracy, allowing for end-to-end optimization of the entire alignment pipeline.
- The Modified Noise Contrastive Estimation (MNCE) loss effectively trains the representation encoder to learn boundary-sensitive acoustic features by distinguishing between stable intra-phoneme regions and transitional boundary frames.
- A model trained exclusively on phoneme-level annotations in one language can effectively generalize to both word-level alignment and unseen languages, indicating the learning of universal acoustic cues for phoneme transitions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.