Reproduce “Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled Criteria” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Joonwon Jang, Jaehee Kim, Wonbin Kweon, Seonghyeon Lee, Hwanjo Yu
The Verbosity-Aware Rationale Reduction (VARR) framework enables Large Language Models to streamline reasoning paths by selectively removing redundant sentences based on a principled, likelihood-based 'verbosity' criterion. This method, extended with VARR+ to contrast with wrong answers, improves average reasoning accuracy by 7.71% and reduces generated tokens by 19.37% across various arithmetic and commonsense tasks.
Key insights
- Initial reasoning sentences in a rationale frequently contain redundancy, allowing for their removal without significantly impacting the correct answer's likelihood.
- Using sentences as reduction units is empirically more effective than tokens, as it better maintains semantic integrity during the pruning process.
- Principled, likelihood-based criteria for rationale reduction are crucial for improving both efficiency and accuracy, outperforming heuristic-driven approaches.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.