Reproduce “Token Turing Machines” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Michael S. Ryoo, Keerthana Gopalakrishnan, Kumara Kahatapitiya, Ted Xiao, Kanishka Rao, Austin Stone, Yao Lu, Julian Ibarz and 1 more
Token Turing Machines introduce a novel Transformer-based architecture that integrates an external, fixed-size memory module to efficiently process arbitrarily long sequential visual inputs with a constant computational cost per step. The model achieved improved performance on online temporal activity detection, spatio-temporal activity detection, and vision-based robot action policy learning tasks.
Key insights
- Formulating memory read and write operations as a unified token summarization process simplifies memory-augmented network design and improves trainability compared to explicit NTM mechanisms.
- Maintaining a fixed-size external memory allows for efficient online processing of infinitely long sequences by ensuring a constant computational cost per inference step.
- Integrating learned positional embeddings helps the model distinguish between current input and historical memory tokens, enabling a robust content- and location-aware memory interaction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.