Reproduce “QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ahmed Wasfy, Omer Nacar, Abdelakreem Elkhateb, Mahmoud Reda, Omar Elshehy, Adel Ammar, Wadii Boulila

QARI-OCR adapts a Multimodal Large Language Model (Qwen2-VL-2B-Instruct) through iterative fine-tuning on custom synthetic datasets to achieve high-fidelity Arabic text recognition. The QARI v0.2 model sets a new open-source state-of-the-art, demonstrating a Character Error Rate (CER) of 0.061 and a Word Error Rate (WER) of 0.160 on diacritically-rich Arabic texts, outperforming existing solutions.

Key insights

  • Task-specific adaptation of general-purpose MLLMs, facilitated by meticulously crafted synthetic datasets, is crucial for achieving high-fidelity OCR in complex and under-resourced scripts like Arabic.
  • A trade-off exists between optimizing for raw text recognition accuracy and enabling structural document understanding; targeted synthetic data can guide models toward either objective.
  • Efficient fine-tuning techniques, such as LoRA and 8-bit quantization, enable the development of highly accurate and specialized models for complex language tasks without requiring prohibitively high computational resources.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.