Reproduce “Learning to Code with Context: A Study-Based Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Uwe M. Borghoff, Mark Minas, Jannis Schopp
Researchers from the University of the Bundeswehr Munich empirically analyzed student use of generative AI in software projects, revealing that while widely adopted, general AI tools struggle with project-specific context; they then developed a repository-aware LLM assistant utilizing Retrieval-Augmented Generation (RAG) that demonstrated improved integration and contextual support.
Key insights
- General-purpose AI tools are widely adopted by students (89.4%) but frequently fail to provide helpful, context-aware advice for project-specific code due to a lack of domain understanding.
- Contextual grounding through Retrieval-Augmented Generation (RAG) significantly mitigates the integration weaknesses of general AI, producing repository-aligned outputs for complex coding tasks.
- While hallucinations and a "retrieval-to-action gap" persist, improving tool utilization and prompt engineering for integrating retrieved context can be more impactful than merely increasing LLM size.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.