Reproduce “Leveraging Language Models to Discover Evidence-Based Actions for OSS Sustainability” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nafiz Imtiaz Khan, Vladimir Filkov
Researchers from the University of California, Davis, developed an LLM-based pipeline to extract 1,312 "Researched Actionables" (ReACTs) from 474 top-tier software engineering papers, transforming academic findings into concrete, evidence-based recommendations for Open Source Software (OSS) maintainers. The study demonstrated how these categorized actionables can be integrated with existing sustainability forecasting tools for practical application.
Key insights
- Large Language Models, specifically Mixtral-8x7B with Chain-of-Thought prompting, can reliably and effectively extract structured, evidence-based actionable recommendations from software engineering literature.
- A substantial catalog of over 1,300 high-quality, evidence-backed actionables exists within the academic literature, offering a significant resource for OSS sustainability.
- The explicitness and completeness of evidence in research findings vary across categories, with technical practices (e.g., CI/CD, testing) generally yielding more explicit ReACTs than socio-technical areas.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.