Reproduce “TFRank: Think-Free Reasoning Enables Practical Pointwise LLM Ranking” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yongqi Fan, Xiaoyang Chen, Dezhi Ye, Jie Liu, Haijin Liang, Jin Ma, Ben He, Yingfei Sun and 1 more
A novel framework, TFRank, enables efficient and practical pointwise LLM ranking by internalizing complex reasoning capabilities within small-scale models during training. This approach reduces inference latency by eliminating explicit Chain-of-Thought generation, yielding performance competitive with larger models on reasoning benchmarks and superior throughput.
Key insights
- Small-scale LLMs can effectively internalize complex reasoning patterns during training, making explicit Chain-of-Thought generation unnecessary and even detrimental to performance during rapid inference.
- The "think-mode switch" successfully decouples an LLM's inherent reasoning ability from the need for verbose output, enabling the direct generation of concise, yet precise, relevance scores.
- Pointwise ranking models can achieve advanced reasoning capabilities comparable to larger, more complex listwise or pairwise methods without sacrificing efficiency, through tailored multi-task training and CoT distillation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.