Reproduce “Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sinchana Ramakanth Bhat, Max Rudat, Jannis Spiekermann, Nicolas Flores-Herr
Researchers at Fraunhofer IAIS systematically analyzed the impact of fixed-size document chunking on retrieval performance for Retrieval-Augmented Generation (RAG) systems across six diverse datasets and two distinct embedding models. The study found that optimal chunk size is not universal, but rather highly dependent on the specific characteristics of the dataset and the architectural biases of the embedding model being used.
Key insights
- There is no universal optimal chunk size; the ideal size is highly dependent on the dataset's characteristics (e.g., answer locality, document structure) and the specific embedding model used.
- Decoder-based embedding models like Stella generally perform better with larger chunk sizes (512-1024 tokens) due to their ability to leverage global context, while encoder-based models like Snowflake maintain competitive performance with smaller chunks.
- Concise, factual answers in structured documents (e.g., SQuAD) benefit from smaller chunks (64-128 tokens), whereas broader, dispersed answers in unstructured or complex documents (e.g., NarrativeQA, TechQA) require larger chunks for effective retrieval.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.