Reproduce “Improving Socratic Question Generation using Data Augmentation and Preference Optimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nischal Ashok Kumar, Andrew Lan

Researchers at the University of Massachusetts Amherst developed a method to improve Socratic question generation for code debugging, leveraging data augmentation and Direct Preference Optimization (DPO) with open-source Large Language Models (LLMs). Their DPO-optimized LLama 2-7B model achieved the highest Rouge-L F1 score of 18.3, outperforming both supervised fine-tuning baselines and demonstrating performance comparable to or superior to proprietary models like GPT-4 in generating pedagogically sound questions.

Key insights

  • Explicitly training LLMs to prefer valid Socratic questions over various types of invalid ones via DPO significantly enhances the pedagogical quality of generated questions, outperforming standard supervised fine-tuning.
  • High-quality negative samples for preference optimization can be effectively created through targeted synthetic data augmentation, including a consistency checking step to ensure data fidelity.
  • Smaller, open-source LLMs can achieve performance comparable to or better than much larger proprietary models in complex educational tasks like Socratic question generation when aligned with human preferences through DPO.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.