Reproduce “TRACT: Regression-Aware Fine-tuning Meets Chain-of-Thought Reasoning for LLM-as-a-Judge” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Cheng-Han Chiang, Hung-yi Lee, Michal Lukasik
The method, TRACT, integrates Chain-of-Thought reasoning with regression-aware fine-tuning through a two-stage self-distillation process for LLM-as-a-judge models. This approach consistently outperforms prior state-of-the-art models like Prometheus-2-7B, achieving an average Pearson's r 0.059 higher on text evaluation benchmarks.
Key insights
- Integrating Chain-of-Thought (CoT) reasoning with regression-aware fine-tuning via a multi-objective CoT-RAFT loss improves numerical text evaluation.
- A two-stage fine-tuning process using self-generated CoTs and re-initialization of the base model effectively aligns the training and inference CoT distributions, which is critical for performance.
- The method demonstrates that a pointwise LLM-as-a-judge model can achieve strong performance on pairwise reward modeling tasks, suggesting a more generalizable understanding of text quality.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.