Reproduce “From PDF to RAG-Ready: Evaluating Document Conversion Frameworks for Domain-Specific Question Answering” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

José Guilherme Marques dos Santos, Ricardo Yang, Rui Humberto Pereira, Alexandre Sousa, Brígida Mónica Faria, Henrique Lopes Cardoso, José Duarte, José Luís Reis and 3 more

Researchers systematically evaluated open-source PDF-to-Markdown conversion frameworks for their impact on Retrieval-Augmented Generation (RAG) question-answering accuracy. They found that optimal preprocessing, including hierarchical splitting and VLM-based image descriptions, led to a 15-percentage-point accuracy span and enabled automated pipelines to surpass manual curation.

Key insights

  • Data preparation quality, specifically document preprocessing, is the dominant factor in RAG system accuracy, leading to a 15-percentage-point performance span across configurations.
  • The splitting strategy and enrichment with structural metadata (e.g., hierarchical splitting with breadcrumbs) are more impactful for RAG accuracy than the choice of conversion framework alone.
  • Naive, unguided knowledge graph construction (GraphRAG) through automated entity extraction may not inherently outperform a meticulously optimized basic RAG pipeline and incurs high computational costs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.