Reproduce “Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wen-Yi Hsiao, Jen-Yu Liu, Yin-Cheng Yeh, Yi-Hsuan Yang

A new music representation called Compound Word (CP) groups related musical tokens into single time steps, combined with a specialized Transformer architecture featuring type-specific prediction heads. This method facilitates the generation of full-length expressive piano music, demonstrating up to 10x faster training and 7x faster inference compared to state-of-the-art Transformer-XL baselines, alongside competitive or enhanced generative quality.

Key insights

  • Explicitly modeling the heterogeneity of musical tokens and their co-occurrence relationships, rather than treating them uniformly, enhances the model's ability to capture musical structure and dependencies.
  • Compressing multiple related individual tokens into a single "compound word" significantly reduces the effective sequence length, addressing the computational bottleneck of Transformers for long musical pieces.
  • The "compound word" representation and its Transformer-based processing can be theoretically framed as learning over discrete-time dynamic directed hypergraphs, offering a new perspective on music sequence modeling.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.