Reproduce “Hypothesis Testing for Generalized Thurstone Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Anuran Makur, Japneet Singh
This paper develops a comprehensive minimax hypothesis testing framework for Generalized Thurstone Models (GTMs), establishing theoretical optimality for detecting deviations from GTMs and providing robust guarantees for sequential data. It includes a novel test statistic and empirical validation, demonstrating its utility in assessing GTM fit in LLM evaluation and sports analytics data.
Key insights
- The critical threshold for detecting deviations from a GTM scales as $O(1/nk)$ for complete observation graphs, establishing the minimax optimality of the proposed test's scaling.
- The test statistic exhibits a reverse martingale property, which enables the derivation of time-uniform upper bounds on Type I and Type II error probabilities, ensuring robustness for sequential data analysis.
- The detectability limits for GTMs are influenced by the observation graph topology, with sparser graphs (e.g., d-regular, cycle, toroidal grid) making deviations harder to detect (requiring larger separation or more data).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.