Reproduce “Requirements Elicitation Follow-Up Question Generation” on OpenResearch

About this paper

Yuchen Shen, Anmol Singhal, Travis Breaux

This research explores leveraging Large Language Models (LLMs) to generate follow-up questions for real-time requirements elicitation interviews. The study finds that minimally guided LLMs produce questions comparable to human interviewers, and when explicitly instructed to avoid common interviewing mistakes, LLM-generated questions are rated significantly superior to human-authored ones, achieving higher scores in relevancy, clarity, and informativeness.

Key insights

• Most human-generated follow-up questions require very limited prior conversational context (typically one to four turns), making real-time LLM-based assistance computationally feasible.
• LLMs can generate follow-up questions that are statistically indistinguishable from those produced by trained human interviewers, even with minimal guidance.
• Explicitly instructing LLMs to avoid common interviewer mistakes leads to the generation of significantly higher quality questions, surpassing human-authored alternatives in error avoidance and overall quality metrics.

Summary by alphaXiv.