Reproduce “MUSE: Revealing Responsiveness Gaps of LLM Tutors on Multiple-Solution Programming Problems” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sky Cen, Shashank Sonkar
The authors introduce MUSE, a framework for testing whether programming tutors follow students’ chosen strategies and recommend alternatives after completion. It controls simulated student approaches across 150 programming problems. In a Gemma probe-first test with correct intuition, adding candidate solutions at inference raises the post-solution transition score from 0.573 to 0.936. The result concerns simulated conversations, not learning by real students.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.