Reproduce “JIR-Arena: The First Benchmark Dataset for Just-in-time Information Recommendation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ke Yang, Kevin Ros, Shankar Kumar Senthil Kumar, ChengXiang Zhai
JIR-Arena frames the task of detecting information needs during activities, retrieving relevant resources, and recommending them at an appropriate time. It provides a benchmark built from 34 lecture and conference scenes and evaluates a text-based baseline. Gemini-2.0-Flash achieved the highest need Recall, 0.429, but a relevance score of 0.014; these prototype results do not establish real-world user benefit.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.