Reproduce “Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhenting Wang, Huancheng Chen, Jiayun Wang, Wei Wei
Accenture's Memex(RL) framework enables large language model agents to tackle long-horizon tasks by maintaining a compact in-context working state linked to a full-fidelity external memory via learned indexing. This approach increased task success on ALFWorld by over 3.5 times, reaching 85.61%, while reducing peak working context length by 43%.
Key insights
- By explicitly indexing and archiving full-fidelity experiences, Memex overcomes the lossy nature of prior context management techniques, allowing unbounded compression without discarding critical evidence.
- Treating memory operations as learnable 'first-class actions' within an RL framework enables agents to develop adaptive and precise memory management strategies tailored to task demands.
- Theoretical analysis confirms that Memex can simultaneously preserve optimal decision quality through bounded, explicit dereferencing, and maintain a bounded working context even as the full interaction history grows infinitely.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.