Reproduce “Short Horizons and Sparse Concepts: a Mathematical View of the Readout in the J-lens” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shi-Qi Yan, Kai-Xuan Ding, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Zhen-Hua Ling

The paper interprets the Jacobian lens as a first-order approximation to expected future readouts, formed by averaging how small activation changes affect later model states. It finds that effects concentrate in short-range, near-diagonal and position-specific pathways, and tests filtering high-energy position pairs. For Qwen3-8B, reported average ICR rises from 18.71 to 22.14, but not on multihop tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.