Table-To-Text generation and pre-training with TabT5
Ewa Andrejczuk, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Yasemin Altun
Abstract
Encoder-only transformer models have been successfully applied to different table understanding tasks, as in TAPAS. A major limitation of these architectures is that they are constrained to classification-like tasks such as cell selection or entailment detection. We present TabT5, an encoder-decoder model that generates natural language text based on tables and textual inputs. TabT5 overcomes the encoder-only limitation by incorporating a decoder component and leverages the input structure with table specific embeddings and pre-training. TabT5 achieves new state-of-the-art results on several domains, including spreadsheet formula prediction with a 15% increase in sequence accuracy, QA with a 2.5% increase in sequence accuracy and data-to-text generation with a 2.5% increase in BLEU.
BibTeX
@inproceedings{andrejczuk-etal-2022-table,
title = "Table-To-Text generation and pre-training with {T}ab{T}5",
author = "Andrejczuk, Ewa and
Eisenschlos, Julian and
Piccinno, Francesco and
Krichene, Syrine and
Altun, Yasemin",
editor = "Goldberg, Yoav and
Kozareva, Zornitsa and
Zhang, Yue",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2022",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.findings-emnlp.503/",
doi = "10.18653/v1/2022.findings-emnlp.503",
pages = "6758--6766"
}