Reproduce “Automated Parsing of Engineering Drawings for Structured Information Extraction Using a Fine-tuned Document Understanding Transformer” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Muhammad Tayyab Khan, Zane Yong, Lequn Chen, Jun Ming Tan, Wenhe Feng, Seung Ki Moon
Researchers from A*STAR and Nanyang Technological University developed a hybrid deep learning framework for automated structured information extraction from 2D engineering drawings. The framework utilizes YOLOv11 for oriented bounding box detection and a fine-tuned Donut transformer, demonstrating superior precision and significantly reduced hallucination rates with a single, generalized model compared to category-specific models.
Key insights
- Combining robust OBB detection with an OCR-free vision-language transformer provides an effective strategy for structured information extraction from the intricate and context-rich domain of engineering drawings.
- A single, generalized Donut model trained across all annotation categories achieves better overall performance, exhibiting higher precision and lower hallucination rates than multiple specialized category-specific models.
- Directly generating structured JSON output enables seamless integration with digital manufacturing systems, significantly reducing manual post-processing and facilitating Industry 4.0 initiatives.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.