Reproduce “Compressing History into Memory: Distilling Transformers into Recurrent Transformers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Philippe Weinzaepfel, Christian Wolf, Mert Bülent Sariyildiz, Guillaume Bono, Gianluca Monaci

Chimera trains a recurrent memory for long-history visual tasks by matching its state to compressed representations from a full-history teacher. On Mem-RPE, it achieves 48% accuracy at sequence length 800, versus 37% for Kinaema, under translation error below 2 m and rotation error below 90°. This supports memory supervision in tested settings, not a single explanation for the performance gap.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.