Reproduce “EviCast: Forecasting Retrieval Gains Through Shared Response Calibration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yibo Wang, Jisheng Dang, Bimei Wang, Jiongpei Hou, Jingwen Zhao, Wenjun Ma, Fei Shen, Juncheng Li and 4 more
EviCast forecasts whether retrieval helps a fixed generator by measuring its responses to controlled evidence states, then weighting those responses by each retriever’s offline state frequencies. On HotpotQA, it matched all 14 deployment decisions with 2.75-point mean absolute error in predicted F1 gain, using 96 diagnostic generation calls versus 480 for Direct, across 12 deployable configurations and two controls.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.