Reproduce “A Mathematical Theory of the Jacobian Lens” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mitchell Ostrow, Daoyuan Qian, Toni Liu, Ila Fiete
The J-lens favors hidden-state directions whose effects are strong and consistent across model depth, future token positions, and prompts. Its strength reflects magnitude, lag coherence, and context coherence, not activation variance alone. In a hidden Markov model, theory predicts a ranking crossover at persistence 0.649; a toy transformer transfers it approximately, with error O(0.01). This validation is limited to toy systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.