Reproduce “LLM$\times$MapReduce: Simplified Long-Sequence Processing using Large Language Models” on OpenResearch

About this paper

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Rongqiao An and 6 more

Researchers from Tsinghua University developed LLM×MapReduce, a training-free framework that enables Large Language Models to process documents far beyond their native context limits by adapting the classic MapReduce paradigm. This framework, which includes a structured information protocol and in-context confidence calibration, outperforms both standard LLMs with larger context windows and other divide-and-conquer methods on long-sequence benchmarks, maintaining high accuracy on documents up to 1.28 million tokens.

Key insights

• The classic MapReduce programming model can be effectively adapted to LLM workflows to handle documents exceeding native context windows.
• A structured information protocol standardizing the output format (extracted information, rationale, answer, confidence) is crucial for managing inter-chunk dependencies and resolving conflicts.
• Implementing in-context confidence calibration through prompt engineering improves the reliability of LLM self-assessment, leading to more informed decisions during information aggregation.

Summary by alphaXiv.