Reproduce “Making Pre-trained Language Models Great on Tabular Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiahuan Yan, Bo Zheng, Hongxia Xu, Yiheng Zhu, Danny Z. Chen, Jimeng Sun, Jian Wu, Jintai Chen
TP-BERTa adapts a pretrained language model for prediction on heterogeneous tables by encoding numerical values as shared, ordered magnitude tokens and combining each value with its feature name before processing rows. Across 80 classification and 65 regression tasks, it achieved average ranks of 5.8 and 4.3, respectively, but performed worse on all-numerical classification datasets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.