Reproduce “Confidence-Calibrated Small-Large Language Model Collaboration for Cost-Efficient Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chuang Zhang, Zizhen Zhu, Yihao Wei, Bing Tian, Junyi Liu, Henan Wang, Xavier Wang, Yaxiao Liu
AWS and Tsinghua University researchers developed COREA, a collaborative framework where a small language model (SLM) is trained to accurately assess its confidence in answers and defer uncertain queries to a larger, more capable language model (LLM). This system achieved up to 21.5% cost reduction with minimal accuracy drops (within 2 percentage points) on reasoning tasks compared to using LLMs alone.
Key insights
- SLMs can be effectively trained to exhibit
- self-awareness
- by accurately calibrating their verbalized confidence in answers.
- The proposed RLCC method simultaneously enhances the SLM's reasoning abilities and its confidence calibration across diverse tasks and model architectures.
- Estimating correctness probability at a group level during RL training is crucial for achieving well-calibrated confidence and preventing overconfidence in SLMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.