Reproduce “CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruchir Puri, David S. Kung, Geert Janssen, Wei Zhang, Giacomo Domeniconi, Vladimir Zolotov, Julian Dolby, Jie Chen and 9 more
CodeNet is a meticulously curated, large-scale dataset for AI for Code research, comprising nearly 14 million code samples in 55 programming languages from competitive programming platforms. It includes rich metadata such as execution status, CPU time, memory usage, and input/output test cases, alongside tools for generating various code representations, aiming to accelerate advancements in tasks like code classification, similarity, and translation.
Key insights
- The availability of large-scale, diverse, and meticulously annotated datasets is fundamental for propelling advancements in the "AI for Code" domain, similar to how ImageNet influenced computer vision.
- Rigorous data cleansing, including identifying and removing near-duplicate code and identical problems, is crucial for building robust AI models that generalize effectively and avoid artificial performance inflation.
- Different code representations (e.g., token sequences, parse trees, code graphs) offer distinct advantages for various AI for Code tasks, with structured graph-based representations proving particularly effective for code similarity and classification.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.