Reproduce “Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zhaowei Wang, Lishu Luo, Haodong Duan, Weiwei Liu, Sijin Wu, Ji Luo, Shen Yan, Shuai Peng and 4 more

MMProLong trains models to retrieve answers from full visual documents by pairing questions generated from sections with source documents. It uses long-document VQA rather than OCR transcription, with a mixture favoring extraction over reasoning. On three benchmarks, it improves the average by 7.11 percentage points over its 7B starting model after 5B training tokens, limited to tested benchmarks and scale.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.