Reproduce “Guiding Language Model Reasoning with Planning Tokens” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xinyi Wang, Lucas Caccia, Oleksiy Ostapenko, Xingdi Yuan, William Yang Wang, Alessandro Sordoni
The paper introduces a structural guidance mechanism for Chain-of-Thought (CoT) reasoning in large language models by enabling them to dynamically generate specialized "planning tokens" before each reasoning step. This approach enhances accuracy across diverse reasoning tasks, achieving an average improvement of 3.3 percentage points with a negligible increase in model parameters.
Key insights
- Machine-learned planning tokens (K-Means, SQ-VAE) consistently and significantly outperform generic prefix tokens or heuristic-based planning, highlighting the need for specialized, information-bearing guidance.
- Specialized planning tokens receive higher attention from the LLM and propagate important contextual information throughout the CoT sequence, indicating their active utilization in guiding reasoning.
- There is an optimal granularity for planning types; too few clusters are too generic, while too many can lead to noisy or indistinguishable planning types for the inference model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.