Reproduce “WebSRC: A Dataset for Web-Based Structural Reading Comprehension” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingyu Chen, Zihan Zhao, Lu Chen, Danyang Zhang, Jiabao Ji, Ao Luo, Yuxuan Xiong, Kai Yu
This work introduces WebSRC, a large-scale, multi-modal dataset and benchmark for Web-based Structural Reading Comprehension, enabling machines to understand web pages by integrating text, HTML structure, and visual layout. Experiments show that incorporating HTML and visual information significantly improves question answering performance compared to text-only approaches, highlighting the necessity of multi-modal understanding for complex web content.
Key insights
- HTML structural information, explicitly provided by tags, is vital for understanding web page content, with models leveraging it significantly outperforming plain-text approaches.
- Visual features from screenshots provide complementary, albeit smaller, gains in comprehension, suggesting that explicit visual cues further aid in structural understanding.
- Web-based Structural Reading Comprehension (SRC) is a significantly more challenging task than traditional text-based Machine Reading Comprehension, as indicated by the lower performance of state-of-the-art models on WebSRC compared to standard benchmarks.
- Models struggle particularly with complex tabular and comparison layouts, highlighting areas where current multi-modal reasoning capabilities need improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.