Reproduce “Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Annalisa Szymanski, Noah Ziems, Heather A. Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, Ronald A. Metoyer
Researchers from the University of Notre Dame and Purdue University investigated the reliability of using LLMs as judges in specialized domains, comparing LLM evaluations with those of human subject matter experts in dietetics and mental health. Their findings indicate limited agreement between LLM judges and experts, with LLMs showing much higher alignment with lay user preferences, suggesting a gap in current LLM evaluation methods for expert knowledge areas.
Key insights
- LLM judges show limited agreement with subject matter experts in specialized domains (dietetics, mental health), particularly when compared to agreement among human experts themselves.
- LLM evaluations align more closely with general user preferences and lay knowledge than with expert standards, suggesting current RLHF methods may not prioritize expert alignment.
- Prompting LLMs with expert personas offers only a modest improvement in agreement with human experts, indicating that persona engineering alone cannot fully bridge the expertise gap.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.