Reproduce “Logit Arithmetic Elicits Long Reasoning Capabilities Without Training” on OpenResearch

About this paper

Yunxiang Zhang, Muhammad Khalifa, Lechen Zhang, Xin Liu, Ayoung Lee, Xinliang Frederick Zhang, Farima Fatahi Bayat, Lu Wang

University of Michigan researchers present THINKLOGIT and THINKLOGIT-DPO, methods that leverage logit arithmetic to elicit complex, long Chain-of-Thought reasoning from large, frozen language models without additional training. This approach uses a substantially smaller guider model and achieves up to 29% relative improvement in mathematical reasoning accuracy over baseline models.

Key insights

• Complex, long-form reasoning capabilities can be elicited from large, frozen language models at inference time by manipulating their output logits using a small, specialized 'guider' model.
• The quality and precision of the guidance signal, particularly through DPO-based alignment, are crucial for performance gains, rather than merely increasing chain-of-thought length or simple imitation learning.
• Large language models may possess latent reasoning abilities that can be 'unlocked' through lightweight decoding-time interventions, without requiring extensive, costly fine-tuning or reinforcement learning.

Summary by alphaXiv.