Reproduce “Enhancing Lexicon-Based Text Embeddings with Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yibin Lei, Tao Shen, Yu Cao, Andrew Yates
LENS builds lexicon-based embeddings by clustering language-model output-token vectors and allowing tokens to attend to full-text context. On MTEB’s 56 datasets across seven categories, LENS-8000 scores 71.63, versus 71.24 for BGE-en-ICL, exceeding it in six categories but trailing on summarization. Their concatenation reaches 63.00 average nDCG@10 on 15 BEIR datasets, improving on both models on 12 of 15.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.