Reproduce “Jev-ToM: Evaluating Theory of Mind in Typed Decision Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kawai Chung, Chunkit Chan, Yuxuan Liu, Yauwai Yim, Xi Yang, Wenbin Hu, Huihao Jing, Chiu Yeung Tang and 4 more
Jev-ToM tests whether typed decision models can select among supplied candidates on English theory-of-mind questions. It compares five typed models and seven generative baselines across nine benchmarks. With controls included, Jev scores 73.2% versus GPT-3.5’s 71.6%; without controls, it scores 69.1% versus 70.9%. Confidence and supplied reasons affect reliability, while complete assistants and interactive task success remain unevaluated.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.