Reproduce “Timo: Towards Better Temporal Reasoning for Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhaochen Su, Jun Zhang, Tong Zhu, Xiaoye Qu, Juntao Li, Min Zhang, Yu Cheng
TIMO introduces a two-stage framework that enhances language models' temporal reasoning across 38 diverse tasks by first establishing a mathematical foundation and then employing a self-critic temporal optimization method. The 7B and 13B TIMO models achieved average accuracy increases of 10.0 and 7.6 scores, respectively, and TIMO-13B outperformed GPT-4 in five out of nine pure-time temporal tasks.
Key insights
- A significant intrinsic correlation exists between temporal reasoning and mathematical reasoning, where strong mathematical abilities can serve as a foundational prerequisite for many temporal tasks.
- Over-tuning with mathematical instructions can lead to performance degradation due to overfitting, highlighting the importance of a balanced and targeted training strategy for specific reasoning types.
- Leveraging an LLM's self-critic capability, combined with a domain-specific hierarchical scoring method, effectively generates high-quality preference data for DPO, which is crucial for generalizing temporal understanding beyond arithmetic.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.