Reproduce “Probing Transformer Representations Layer by Layer, Module by Module” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ambroise Odonnat, Theo Gnassounou, Laetitia Chapel, Romain Tavenard, Ievgen Redko

The authors study which transformer features support linear-probe classification, comparing representations from eight components across block depths. They find that the best readout depends on both component and depth: fixed Act@100 for vision and LN2@60 for language improve on the final residual-stream baseline in a majority of evaluations. These heuristics recover 41.9% and 86.0% of oracle gains, respectively, among settings evaluated.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.