Reproduce “Using Word Embeddings for Automatic Query Expansion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dwaipayan Roy, Debjyoti Paul, Mandar Mitra, Utpal Garain
Researchers at the Indian Statistical Institute investigate using word2vec embeddings for automatic query expansion in information retrieval, demonstrating improved retrieval effectiveness over unexpanded baselines. They find that incorporating query term bigrams significantly boosts performance, though the method remains less effective than established statistical approaches like RM3.
Key insights
- Representing query bigrams via summed word embeddings consistently enhances the effectiveness of embedding-based query expansion.
- Word embedding-based query expansion methods consistently and significantly improve retrieval effectiveness when compared to an unexpanded Language Model baseline.
- The direct application of word embeddings for query expansion, even with different variants, does not achieve the same level of effectiveness as established statistical pseudo-relevance feedback methods like RM3.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.