Reproduce “Grading Handwritten Engineering Exams with Multimodal Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Janez Perš, Jon Muhovič, Andrej Košir, Boštjan Murovec
Researchers at the University of Ljubljana developed an end-to-end workflow utilizing multimodal large language models to grade scanned handwritten engineering exams, including those with diagrams. The system achieves a mean absolute difference of approximately 8 points compared to human lecturer grades and an estimated 17% manual review rate for strong model backends, significantly automating the assessment process.
Key insights
- Structured prompting, reference grounding (via text-extracted summaries), and supervisor aggregation are critical for achieving reliable, calibrated grading performance and preventing systematic biases.
- Ensemble grading combined with a supervisor LLM significantly enhances robustness and reduces variance, allowing for practical deployment with an auditable manual review trigger rate.
- State-of-the-art multimodal LLMs (e.g., GPT-5.2, Gemini-3 Pro) can achieve high agreement with human graders on complex handwritten STEM content when integrated into a carefully engineered workflow.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.