Reproduce “GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning” on OpenResearch

About this paper

Zhaoxin Yu, Qi Shen, Hengli Li, Zhaowei Zhang, Song-Chun Zhu, Chi Zhang, Zilong Zheng

GradCuit enhances test-time reasoning in large language models by directly inserting optimizable latent states into an intermediate Transformer layer, leveraging self-attention for direct gradient flow from generated continuation tokens. This approach improves average accuracy by 6.6 percentage points over Chain-of-Thought and 2.4 points over LatentSeek, while increasing robustness to hyperparameters and providing insights into the model's internal reasoning dynamics.

Key insights

• Directly integrating optimizable latent states into intermediate Transformer layers and leveraging self-attention for gradient flow resolves the credit-assignment problem and enables more effective optimization.
• The optimization process is robust to learning rate variations and can even achieve competitive performance with random-walk optimization, suggesting an inherently accessible latent space.
• Optimized latent states primarily influence "reasoning connector" tokens, indicating a targeted steering of the model's logical progression and providing a lens for interpreting latent dynamics.

Summary by alphaXiv.