Reproduce “Representations in Motion: Layer-wise Rotation and Trajectory Divergence in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuonan Yang, Yik Siu Chan, Wannan Yang, Ellie Pavlick
The authors track how factuality directions change across layers in one forward pass. They compare true-versus-false activation contrasts under honest and lying instructions, finding early alignment followed by near-opposition. For Llama-3.1-8B-Instruct, the rotation metric is 1.447 ± 0.235 across 148 prompt variants (p < 10⁻⁴). Other tasks show less consistent support; rotation is not shown to cause behavior or serve as a validated detector.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.