Reproduce “LeakageDetector: An Open Source Data Leakage Analysis Tool in Machine Learning Pipelines” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Eman Abdullah AlOmar, Catherine DeMario, Roger Shagawat, Brandon Kreiser
LeakageDetector is an open-source PyCharm plugin designed to identify data leakage in machine learning code by integrating a Dockerized static analysis tool, providing in-IDE feedback and actionable quick fix suggestions. The tool aims to enhance ML model reliability and generalization by making leakage detection more accessible and integrated into developer workflows.
Key insights
- Preprocessing leakage was identified as the most frequently encountered type (55.6%) by users, suggesting its common occurrence in ML development practices.
- The integration of a robust static analysis tool into an IDE significantly improves usability and developer engagement compared to standalone command-line tools.
- While basic quick fixes can be provided, fully automated, semantically correct code corrections for data leakage remain challenging and often require human conceptual understanding.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.