Reproduce “TempoFit: Plug-and-Play Layer-Wise Temporal KV Memory for Long-Horizon Vision-Language-Action Manipulation” on OpenResearch

About this paper

Jun Sun, Boyu Yang, Jiahao Zhang, Ning Ma, Chencheng Wu, Siqing Zhang, Yiou Huang, Qiufeng Wang and 2 more

TempoFit, from Xi'an Jiaotong-Liverpool University, introduces a training-free, plug-and-play module that enhances Vision-Language-Action (VLA) policies with temporal memory by reusing the backbone's internal attention state. It achieves up to a 4.0% absolute improvement in success rate on the LIBERO-LONG benchmark and a 9.5% average improvement on real-world manipulation tasks, with minimal latency and memory overhead.

Key insights

• Transformer's internal K/V pairs can serve as an efficient, model-native content-addressable memory for temporal context, and their stable integration into a frozen VLA backbone requires norm-preserving residual loading.
• Applying a recency-biased retrieval (Frame-Gap Temporal Bias) is crucial for effective training-free temporal context injection, ensuring present-dominance and preventing reliance on stale information.
• Optimal temporal information resides in selected intermediate Transformer layers, with selective caching outperforming memory activation across all layers or only extreme layers.

Summary by alphaXiv.