Reproduce “Applying Large Language Models and Chain-of-Thought for Automatic Scoring” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Gyeong-Geon Lee, Ehsan Latif, Xuansheng Wu, Ninghao Liu, Xiaoming Zhai
The paper evaluates GPT-4 for scoring written science explanations by prompting it with human-scored examples, assessment context, rubrics, and chain-of-thought reasoning. Across 1,650 responses from six tasks, zero-shot chain-of-thought with context and rubrics averaged .6831 accuracy, versus .5487 without chain-of-thought, a reported 13.44% relative increase. Results varied by task, and supervised ensemble models reported higher accuracy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.