Reproduce “LLaMa-SciQ: An Educational Chatbot for Answering Science MCQ” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marc-Antoine Allard, Matin Ansaripour, Maria Yuffa, Paul Teiletche
LLaMa-SciQ adapts an existing language model through math-and-coding training, preference tuning, and multiple-choice-question training to produce answers with explanations. It scores 74.5% accuracy on GSM8K in a zero-shot evaluation, compared with 79.6% for an instruction model using eight-shot chain-of-thought prompting. On the roughly 350-question EPFL STEM benchmark, it scores 45.21%; retrieval and 4-bit quantization score lower.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.