Reproduce “TIDE: Tree-Induced Fine-Grained Diagnostic Evaluation from Model Behaviors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiayi Chen, Hui Li, Yujiong Shen, Qiyuan Peng, Yanxin Li, Chenhao Huang, Shaofan Liu, Guoqiang Zhang and 7 more
TIDE uses benchmark results to build question hierarchies that diagnose where models differ. An LLM proposes question groupings, while observed scores select splits that balance within-model consistency and between-model separation. On MMLU-Pro, GPT-5.4’s and Claude Opus 4.6’s overall accuracies are 85.2% and 86.2%, but slice results reverse across capabilities. This evidence is limited to static, single-turn English benchmarks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.