Reproduce “From a Large-Scale Teacher to Efficient Multimodal Students for Search Relevance Prediction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tomasz Gniazdowski, Mohammadreza Momenifar, Nadia Vase, Shahram Khadivi, Wiktor Walentynowicz, Nicola Ueffing, Jan Pawlowski, Piotr Szymanski and 1 more

The authors train smaller multimodal models to judge e-commerce query-item relevance using labels and explanations generated by a prompted GPT-4.1, calibrated against human judgments. The 12B Gemma-3-IT student reaches 65.24% accuracy and 57.56% macro-F1 on an independent test of 233 pairs, versus GPT-4.1’s 64.38% and 56.09%. This supports comparable performance on that small, single-annotated test, not universal superiority.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.