Reproduce “Could ChatGPT get an Engineering Degree? Evaluating Higher Education Vulnerability to AI Assistants” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Beatriz Borges, Negar Foroutan, Deniz Bayazit, Anna Sotnikova, Syrielle Montariol, Tanya Nazaretzky, Mohammadreza Banaei, Alireza Sakhaeirad and 82 more
The authors evaluate GPT-3.5 and GPT-4 on 5,579 English–French text-only questions from 50 EPFL courses, using eight prompting strategies. They score multiple-choice answers and assign partial credit to open answers. GPT-4 reaches 65.8% under majority aggregation; its best result across strategies is 85.1%. At a 50% cutoff, 89% of sampled courses meet GPT-4’s MCQ threshold, not official pass rates.
Key insights
- Higher education assessment schemes, particularly in engineering and natural sciences, face immediate vulnerability to student utilization of generative AI tools, posing a credible short-term risk.
- Assessments require urgent re-evaluation and revision to become 'AI-adversarial,' emphasizing analytical and applied knowledge, and considering a return to proctored examinations for factual recall.
- Education must adapt pedagogical approaches to foster critical thinking, prevent circumvention of foundational learning, and educate students on the ethical use of AI systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.