Reproduce “MX-Font++: Mixture of Heterogeneous Aggregation Experts for Few-shot Font Generation” on OpenResearch

About this paper

Weihang Wang, Duolin Sun, Jielei Zhang, Longwen Gao

Researchers at BILIBILI Inc. developed MX-Font++, a few-shot font generation model that enhances content-style disentanglement and feature extraction using Heterogeneous Aggregation Experts and a content-style homogeneity loss. The model achieves state-of-the-art visual quality for complex Chinese characters and substantially improves scene text recognition accuracy for low-resource languages like Cyrillic when trained with its generated fonts.

Key insights

• The combination of Transformer-based attention mechanisms within a Mixture of Experts framework, specifically through Heterogeneous Aggregation Experts, significantly enhances the model's ability to capture both local and global features in complex character glyphs.
• Explicitly enforcing disentanglement between content and style features through a dedicated homogeneity loss in the latent space leads to more controllable, consistent, and high-fidelity font generation.
• High-quality synthetic font generation, particularly for complex and low-resource languages, can substantially improve the performance of downstream artificial intelligence applications like multilingual scene text recognition.

Summary by alphaXiv.