Reproduce “PixT3: Pixel-based Table-To-Text Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Iñigo Alonso, Eneko Agirre, Mirella Lapata
PixT3 recasts table-to-text generation as a visual recognition task, processing tables as images rather than linearized text. This pixel-based approach achieves nearly double the BLEU and PARENT scores of text-only models in loosely controlled generation on the ToTTo dataset and significantly improves faithfulness, maintaining robustness across varying table sizes.
Key insights
- Recasting table-to-text as a visual recognition task preserves the 2D structural information of tables, leading to improved understanding and generation.
- Processing tables as images provides a robust method for handling varying table sizes and layouts, mitigating the limitations of long textual sequences.
- A self-supervised structure learning curriculum can effectively instill table structure awareness into general-purpose vision models, improving faithfulness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.