Reproduce “OCR-Memory: Optical Context Retrieval for Long-Horizon Agent Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinze Li, Yang Zhang, Xin Yang, Jiayi Qu, Jinfeng Xu, Shuo Yang, Junhua Ding, Edith Cheuk-Han Ngai
OCR-Memory introduces a memory framework for autonomous LLM agents that optically encodes interaction histories into visual tokens, enabling complete retention of long past experiences and deterministic, hallucination-free retrieval. This approach achieves higher task success rates and maintains high retrieval accuracy under strict token budgets compared to text-based methods.
Key insights
- Visual encoding provides a high-density, loss-free medium for retaining extensive agent interaction histories, effectively bypassing LLM context window limitations.
- Decoupling context understanding from evidence generation through index-based optical retrieval significantly enhances retrieval faithfulness and reduces hallucination compared to free-form text generation.
- Mimicking human memory's "vivid-to-fuzzy" property via multi-resolution storage and active recall enables efficient management of visual-token costs while preserving access to critical, high-fidelity information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.