Reproduce “LANGSAE EDITING: Improving Multilingual Information Retrieval via Post-hoc Language Identity Removal” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dongjun Kim, Jeongho Yoon, Chanjun Park, Heuiseok Lim
LANGSAE EDITING introduces a post-hoc sparse autoencoder pipeline that suppresses language-identity signals within multilingual embeddings to enhance cross-language information retrieval. This approach significantly boosts macro-average nDCG@20 by +21.9% on Belebele and achieves a computational efficiency 18.4 times greater than re-running the base encoder.
Key insights
- Language identity signals are concentrated within specific, isolable sparse latent features in multilingual embeddings, which can be precisely targeted for suppression.
- The suppression of these identified language-associated features directly reduces 'same-language crowding' and significantly improves cross-lingual retrieval performance.
- A post-hoc, vector-only approach using sparse autoencoders offers a practical and efficient solution for mitigating bias in existing retrieval infrastructures without requiring costly re-training or re-encoding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.