Reproduce “SMART: State-based Multi Agent Reasoning Traces for Verification-oriented Problem-Solving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sai Vallurupalli, Francis Ferraro
SMART helps judge scientific claims by making relevant background relationships explicit before reaching a verdict. It assigns entity-focused agents to generate contrasting premises, checks their truth and stance, and gives selected premises to a reasoning model. On SFBench, Qwen3 32B with SMART-PK reaches balanced accuracy 0.569 versus 0.362 for chain-of-thought after the Refute Filter, which excludes unsupported refutations from scoring.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.