Reproduce “Automated Genre-Aware Article Scoring and Feedback Using Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chihang Wang, Yuxin Dong, Zhenhong Zhang, Ruotong Wang, Shuo Wang, Jiajing Chen
A new automated article scoring framework from New York University, Wake Forest University, and Purdue University incorporates genre-specific characteristics and generates personalized feedback. The system achieves a Quadratic Weighted Kappa (QWK) index of 0.803, outperforming traditional methods in evaluating diverse writing.
Key insights
- Integrating genre-specific characteristics into automated essay scoring (AES) significantly improves scoring accuracy and relevance compared to universal criteria.
- Combining advanced NLP techniques like BERT and LSTM with large language models (ChatGPT-3.5) enables both accurate scoring and actionable, personalized feedback generation.
- Providing detailed, trait-specific feedback transforms AES from a mere assessment tool into an active educational resource, supporting learning and improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.