Reproduce “A Survey on Data Cleaning Methods for Improved Machine Learning Model Performance” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ga Young Lee, Lubna Alzamil, Bakhtiyar Doskenov, Arash Termehchy
This paper reviews five state-of-the-art data cleaning methods introduced after 2015, evaluating their efficacy in improving machine learning model performance. It identifies pervasive issues including the absence of optimal cleaning stopping points, a fundamental tension between cleaning efficiency and data coverage, and insufficient generalizability across diverse datasets and models, proposing future research avenues.
Key insights
- Current advanced data cleaning methods often lack automated optimizers to determine ideal stopping points, requiring significant manual parameter tuning and domain expertise.
- A fundamental tension exists between achieving comprehensive data cleaning coverage and maintaining computational efficiency, especially in large-scale databases.
- Many reviewed methods exhibit limited generalizability, performing effectively only on specific data types, error patterns, or machine learning models.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.