Reproduce “mPLUG-PaperOwl: Scientific Diagram Analysis with the Multimodal Large Language Model” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anwen Hu, Yaya Shi, Haiyang Xu, Jiabo Ye, Qinghao Ye, Ming Yan, Chenliang Li, Qi Qian and 2 more

PaperOwl adapts a document-understanding model for scientific diagram analysis by combining preceding text, figures or tables, and a brief outline of points to address. The authors train it on M-Paper examples linking diagrams with captions and analysis. It scores CIDEr_gpt 11.50 versus DocOwl’s 1.23, but the metric is not factual verification, and examples contain numerical errors.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.