Reproduce “Uncovering bias in the PlantVillage dataset” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mehmet Alican Noyan
A study quantitatively demonstrates significant dataset bias in the widely used PlantVillage dataset, revealing that a simple model trained on only eight background pixels achieved 49.0% classification accuracy, far exceeding the 2.6% random chance baseline. This work indicates that machine learning models can exploit spurious correlations related to image capture conditions rather than learning true disease features.
Key insights
- The PlantVillage dataset contains substantial exploitable bias, where background pixels alone are highly correlated with class labels.
- Capture bias, stemming from the image acquisition process, can affect both foreground and background elements and is not fully mitigated by background removal.
- Many reported high accuracies for PlantVillage-trained models likely reflect learning superficial cues rather than robust disease characteristics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.