Reproduce “CaRT: Teaching LLM Agents to Know When They Know Enough” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Grace Liu, Yuxiao Qu, Jeff Schneider, Aarti Singh, Aviral Kumar
Carnegie Mellon University's CaRT (Counterfactuals and Reasoning for Termination) framework enables large language models to adaptively determine when to stop gathering information or reasoning, improving task success while reducing resource consumption. This method yielded higher diagnostic accuracy and more optimal termination points in interactive medical diagnosis, and achieved greater accuracy with fewer reasoning tokens in mathematical problem-solving compared to standard baselines.
Key insights
- Generating hard negative counterfactuals is crucial for teaching LLMs to identify the *causal* factors for information sufficiency, helping them avoid terminating based on spurious correlations.
- Verbalized reasoning traces serve as an "implicit value function," enabling LLMs to comparatively weigh the utility of available and missing information, leading to more generalizable and stable termination decisions.
- Both counterfactual training and reasoning augmentation are indispensable; ablation studies demonstrate that removing either component substantially degrades the model's adaptive termination performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.