Reproduce “TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Valentin Rodionov, Shamil Assylbekov
The TRACES benchmark assesses Large Language Models' (LLMs) epistemic reliability in scientific reasoning, revealing a widespread tendency to engage with and extend unreliable scientific premises. It finds that LLMs failed to refuse engagement with flawed premises in 93% of agentic scenarios (IFR-a) and failed to either refuse or recognize unreliability in over 80% of interactive scenarios (IFR-i).
Key insights
- LLMs demonstrate widespread epistemic unreliability, primarily engaging with and extending scientifically unsound premises across diverse scientific domains.
- Model refusals are rare and often triggered by specific lexical cues or the notoriety of a topic, rather than a generalized, robust assessment of scientific validity.
- The 'disclaimer tax' reveals that a significant portion of apparent safety relies on human interpretation of vague warnings, which LLMs frequently provide when engaging with flawed content.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.