Reproduce “CI at Scale: Lean, Green, and Fast” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dhruva Juloori, Zhongpeng Lin, Matthew Williams, Eddy Shin, Sonal Mahajan
Uber engineers enhanced the SubmitQueue continuous integration system with a refined probabilistic model and machine learning-driven build time prediction, achieving approximately 53% reduction in CI resource usage, 44% lower CPU consumption, and 37% faster P95 waiting times for code changes across its monorepos.
Key insights
- Integrating probabilistic build time predictions, including variance, into CI scheduling algorithms can significantly improve resource efficiency and throughput.
- To fully leverage features like 'Bypassing Large Diffs', the CI system needs to intelligently prioritize and execute multiple speculative build paths for a single change, not just one.
- A dynamically determined 'speculation threshold' can effectively prune low-probability builds, leading to substantial reductions in wasted CI resources without impacting change landing rates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.