Reproduce “PURE: A Diagnostic Benchmark for Multi-Granularity Multimodal Identity Matching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hengyu You, Xintong Wang, Jin Cao, Liang Ding
PURE separates base-level matching of underlying designs from variant-level matching, and evaluates multimodal language models on evidence acquisition, attribute understanding, comparison, and identity-rule judgment. Removing titles improved all ten models’ accuracy on one base-level rule task, but reduced accuracy for a different-product category. The benchmark has 1,457 Chinese questions; generalization beyond e-commerce remains unestablished.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.