Reproduce “xPRT: Cross-Passage Retrieval Pretraining” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Danush Khanna, Himanshu Singhal, Pranjal A. Chitale, Bishal Santra
xPRT trains dense retrievers from unlabeled documents by pairing representative passages with related passages elsewhere in each document. It aligns embeddings and predicts masked content across passages, with a frozen E5 encoder and lexical signals curating groups. On MTEB English v2, continued pretraining from E5-base-unsupervised reaches 54.86 average versus 53.14 for a baseline, within the English FineWeb setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.