Reproduce “Invariance before structure: two modes of emergent compression in representation learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hardik Rajpal, Dan Goodman
The authors use EARL to compare whether representations become less dependent on network settings and whether they become self-predictive. In a fixed-reservoir model trained to predict time series, parameter-space compression reaches half its change around epoch 8, versus around epoch 40 for state-space compression. Target controls share early changes, but not later ones; evidence is limited to this setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.