Reproduce “Evaluating the Performance of Large Language Models for SDG Mapping (Technical Report)” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hui Yin, Amir Aryani, Nakul Nambiar
Researchers at Swinburne University of Technology demonstrated that open-source large language models such as LLaMA 3, Mixtral, and Qwen2 can achieve performance for Sustainable Development Goal mapping comparable to proprietary models, while also addressing data privacy and cost concerns.
Key insights
- Open-source LLMs can perform SDG mapping comparably to proprietary models, offering a viable alternative for institutions.
- Different LLMs exhibit distinct precision and recall profiles, allowing selection based on specific application needs (e.g., prioritizing correct classifications vs. comprehensive coverage).
- Deploying open-source LLMs locally addresses critical concerns regarding data privacy and significantly reduces operational costs compared to proprietary services.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.