Reproduce “Embedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models” on OpenResearch

About this paper

Chengkai Huang, Yu Xia, Rui Wang, Kaige Xie, Tong Yu, Julian McAuley, Lina Yao

This research introduces Embedding-Informed Adaptive Retrieval-Augmented Generation (EI-ARAG), an approach for Large Language Models (LLMs) that uses pre-trained token embeddings to efficiently determine when to retrieve external knowledge. The method achieves high accuracy while reducing the rate of retrieval; for instance, on the PopQA dataset with LLaMA 2 (7B), it reached 33.08% accuracy with a 57.89% retrieval rate, significantly improving computational efficiency compared to prior adaptive strategies.

Key insights

• Contextualized pre-trained token embeddings from the early layers of LLMs inherently capture sufficient information about the model's knowledge to effectively guide retrieval decisions.
• Using a small, dedicated classifier on these embeddings provides a computationally efficient alternative to prompting the LLM itself for retrieval decisions, avoiding extra full LLM inference calls.
• Adaptive retrieval based on embedding-informed decisions allows for a better balance between answer accuracy and retrieval frequency, preventing performance degradation from unnecessary or noisy external information.

Summary by alphaXiv.