Reproduce “Measuring In-Context Computation Complexity via Hidden State Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Vincent Herrmann, Róbert Csordás, Jürgen Schmidhuber
Herrmann, Csordás, and Schmidhuber propose PHi (Prediction of Hidden states) loss, a new metric that quantifies the computational complexity of in-context computations in neural sequence models by measuring the unpredictability of internal states. This metric effectively distinguishes complex in-context learning from memorization, correlates with language complexity and mathematical reasoning performance, and provides insights into LLM internal processes.
Key insights
- The experiments revealed distinct computational signatures for different types of tasks, clearly differentiating between memorization and genuine computation within LLMs.
- Results suggest that in-context learning operates through complex computation rather than simple pattern matching, as evidenced by its distinctly high PHi loss.
- During complex reasoning, LLM hidden states follow paths that are structured but not easily predictable from past states, indicating non-trivial computation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.