Reproduce “CulturalRiddles: A Multicultural Benchmark of Riddles” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
My Chiffon Nguyen, Tianyi Hu, Vinod Anbalagan, Henry Gagnier, Pouya Sadeghi, Farah Abdou, Tidor-Vlad Pricope, Diego MMS Oliveira and 156 more
CulturalRiddles tests whether models solve community-authored cultural riddles by combining indirect clues with local associations, rather than recalling direct facts. Native speakers review the riddles. Accuracy is the share of scored answers judged fully correct. Gemini 3.1 Pro Preview reaches 79.4% overall, but no tested model exceeds 22% in Lumasaba or Mongondow, so aggregates conceal community-specific gaps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.