Reproduce “TAPAS: Weakly Supervised Table Parsing via Pre-training” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jonathan Herzig, Paweł Krzysztof Nowak, Thomas Müller, Francesco Piccinno, Julian Martin Eisenschlos

Google Research introduces TAPAS, a model that extends the BERT architecture with table-aware embeddings and large-scale pre-training to perform question answering over tables by directly predicting answers without generating explicit logical forms. It achieved an 11 percentage point improvement in accuracy on the SQA conversational QA dataset and established new state-of-the-art results on WIKITQ through transfer learning.

Key insights

  • Leveraging BERT-style pre-training, adapted for hybrid text-table data, is crucial for learning effective representations for table understanding.
  • Directly predicting answer denotations (cell selection and aggregation) from weak supervision, without intermediate logical forms, simplifies the model and training process while maintaining high performance.
  • Table-aware positional embeddings, especially for column and row indices, are fundamental for the model to effectively parse and reason over tabular structures.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.