Reproduce “GitEvo: Code Evolution Analysis for Git Repositories” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Andre Hora
GitEvo is an agentic system that utilizes structural code analysis and Retrieval-Augmented Generation (RAG) with a critic-guided self-correction loop to automatically update software documentation. This approach, implemented with a fine-tuned Phi-3 Mini model, achieved substantially higher semantic alignment (0.969 summary exact match) and overall quality (3.44 LLM-as-a-Judge score) compared to a CodeT5-base baseline (0.188 summary exact match, 1.91 Judge score) on a documentation repair task.
Key insights
- A unified tool that combines Git-level historical traversal with detailed code-level structural analysis simplifies and enhances the process of understanding software evolution.
- Leveraging robust multi-language parsing libraries like Tree-sitter enables comprehensive code evolution analysis across diverse programming languages within a single framework.
- An extensible API empowers researchers and practitioners to define custom metrics, tailoring evolution studies to specific questions or project needs beyond predefined measurements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.