Reproduce “Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
A. Said Gurbuz, Ahmed Nassar, Christoph Auer, Maksym Lysak, Lucas Morin, Matteo Omenetti, Tim Strohmeyer, Panagiotis Vagenas and 3 more
IBM Research Zurich presents an OCR-free, end-to-end approach for extracting key-value structures directly from document images. A 256-million-parameter SmolDocling model jointly generates text, bounding boxes, semantic roles, and directed links in DocTags format, reaching layout-aware SER/RE scores of 0.514/0.275 on FUNSD, 0.412/0.206 on XFUND, and 0.485/0.321 on DocLayNetV2 while running over five times faster than Qwen2.5-VL 7B.
Key insights
- Combining identification, localization, and association in one structured generation task can produce spatially verifiable outputs while avoiding dependence on external OCR.
- Graph-based crops and synthetic field completion are especially useful for improving relation extraction; expanding graph crops raised the reported XFUND RE result from 0.122 to 0.172.
- Entity recognition is consistently easier than relation prediction, with errors concentrated in distant or repeated links, multi-word boundaries, reading-order mismatches, and deeply nested graphs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.