Reproduce “Benchmarking In-the-wild Multimodal Disease Recognition and A Versatile Baseline” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Tianqi Wei, Zhi Chen, Zi Huang, Xin Yu

Researchers from The University of Queensland developed PlantWild, the largest in-the-wild multimodal dataset for plant disease recognition, featuring 18,542 images across 89 classes with GPT-3.5 generated textual descriptions. They also introduced Multimodal Versatile Plant Disease Recognition (MVPDR), a vision-language model-inspired baseline leveraging multiple visual and textual prototypes, which achieved superior performance on real-world datasets, outperforming existing methods by up to 4.54% in accuracy on PlantDoc.

Key insights

  • The new PlantWild dataset establishes a more realistic and challenging benchmark for plant disease recognition, revealing that existing state-of-the-art methods perform significantly worse on real-world images compared to laboratory-controlled data.
  • Modeling both large intra-class variance and small inter-class discrepancy in plant diseases is effectively achieved by using multiple multimodal prototypes for each class, leveraging both visual features and rich textual descriptions.
  • The strategic integration of LLM-generated textual descriptions with Vision-Language Models provides crucial semantic cues, enabling improved differentiation between visually similar diseases and better characterization of diverse disease manifestations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.