Reproduce “Multi-Axis Speech Similarity via Factor-Partitioned Embeddings” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jim O'Regan, Jens Edlund
Researchers at KTH Royal Institute of Technology introduced a factor-partitioned embedding framework that enables controllable multi-axis speech similarity by dynamically weighting attributes like semantic content, speaker identity, and dialect. This framework demonstrated that a speaker auxiliary task is essential for stable semantic learning, achieving 65.5% P@1 for cross-corpus semantic retrieval with effective speaker bias suppression.
Key insights
- A speaker auxiliary task is not merely helpful but required for the semantic axis to function effectively, preventing mode collapse and providing crucial geometric conditions for stable semantic learning from the frozen acoustic encoder.
- The use of signed weighting on per-axis cosine scores enables a novel "preference-flip" mechanism, allowing the framework to suppress unwanted attributes like same-speaker bias (e.g., with a negative speaker weight).
- PCA reduction of axis dimensions can introduce directional asymmetry, breaking the geometric symmetry necessary for signed axis weights to be interpretable and reversible.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.