Reproduce “Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Thomas Merth, Qichen Fu, Mohammad Rastegari, Mahyar Najibi

Researchers from Apple and Meta developed Superposition Prompting, a RAG framework that uses a Directed Acyclic Graph (DAG) to structure inputs, enhancing both accuracy and inference speed without model fine-tuning. The method yielded up to a 93.7x theoretical speedup and a 43% accuracy increase on NaturalQuestions-Open with MPT-7B-Instruct by leveraging path pruning, caching, and parallelization.

Key insights

  • Representing RAG prompts as a DAG allows for a generalized attention mechanism that improves efficiency and accuracy by effectively reducing the LLM's perceived maximum sequence length from O(N) to O(1).
  • A novel path equilibrium positioning strategy, which assigns real-valued token positions by linearly spacing overlapping paths, significantly outperforms simple left-aligned positioning by avoiding discontinuities.
  • A Bayesian saliency metric is highly effective in identifying and pruning irrelevant documents, directly mitigating the 'distraction phenomenon' and improving LLM output quality.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.