Reproduce “AdaParse: An Adaptive Parallel PDF Parsing and Resource Scaling Engine” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Carlo Siebenschuh, Kyle Hippe, Ozan Gokdemir, Alexander Brace, Arham Khan, Khalid Hossain, Yadu Babuji, Nicholas Chia and 5 more
AdaParse is an adaptive engine designed to parse scientific PDF documents at scale, dynamically selecting the optimal parser for each document to balance text extraction quality and computational efficiency. The system achieved 0.2% better accuracy and a 17x higher throughput compared to existing high-quality parsers, enabling the creation of large, high-quality scientific text datasets for training language models.
Key insights
- No single PDF parser performs optimally across all scientific document types due to their diverse structures and layouts, necessitating an adaptive approach.
- Traditional accuracy metrics for text extraction (e.g., BLEU) do not always align with human perception of parsing quality, highlighting the importance of incorporating user preferences.
- Efficient resource utilization, including parallel execution and hardware acceleration, is crucial for scaling high-quality PDF parsing to massive scientific document collections.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.