Reproduce “AgriBench: A Hierarchical Agriculture Benchmark for Multimodal Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yutong Zhou, Masahiro Ryo
AgriBench presents the first hierarchical benchmark for evaluating Multimodal Large Language Models (MM-LLMs) in agricultural contexts, spanning 17 tasks across five complexity levels. It introduces MM-LUCAS, a novel multimodal dataset of 1,784 agricultural landscape images from the EU, enriched with comprehensive annotations to address data scarcity. Initial evaluations indicate MM-LLMs grasp general agricultural concepts but require specialized fine-tuning for expert-level problems.
Key insights
- AgriBench's hierarchical structure effectively assesses MM-LLMs across varying levels of multimodal and task complexity, providing a granular framework for evaluating agricultural intelligence beyond generic capabilities.
- The MM-LUCAS dataset, built upon the authoritative LUCAS survey and enriched with diverse annotations, provides an unprecedented resource that addresses critical data limitations for MM-LLM development in agriculture.
- Analysis of MM-LUCAS revealed that natural elements like orchards and flower fields significantly contribute to the aesthetic appeal of agricultural landscapes, with notable regional and seasonal variability in image quality and aesthetics across the EU.
- Current mainstream MM-LLMs demonstrate a strong understanding of general agricultural content and contextual scene analysis but struggle with specific, expert-level problems like disease diagnosis without domain-specific fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.