Reproduce “Closing Cost-Quality Gap in Document VLMs: Difficulty-Aware Data Curation and Quality-Adjusted Deployment Economics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maksim Evdokimov, Matvey Ivanov, Dmitrii Tsiupin, Olga Tsymboi, Anatolii Potapov, Aleksandr Ivanov
An on-premise Vision-Language Model (VLM) for document understanding, developed by T-Tech, integrates a Difficulty-Aware Data Curation (DADC) pipeline and a quality-adjusted cost analysis to address challenges in regulated industries. The system achieves over 80% cost reduction compared to human baselines and more than 50% against competing open-source models, maintaining production-level quality and strict latency requirements on a single H100 GPU.
Key insights
- Raw, unfiltered open-domain data can degrade VLM performance, while a structured DADC pipeline is essential to transform it into effective, quality-filtered supervision that positively scales performance.
- Economic viability, quantified through a quality-adjusted cost analysis, is critical for real-world VLM deployment in regulated industries, often outweighing raw benchmark performance.
- Effective SFT and data curation on an MoE architecture can achieve production-level quality and cost-efficiency, outperforming larger reasoning-mode models under strict latency and hardware constraints.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.