GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing
Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, bailin wang, Yi Chern Tan, Xinyi Yang, Dragomir Radev, richard socher
Abstract
We present GraPPa, an effective pre-training approach for table semantic parsing that learns a compositional inductive bias in the joint representations of textual and tabular data. We construct synthetic question-SQL pairs over high-quality tables via a synchronous context-free grammar (SCFG). We pre-train our model on the synthetic data to inject important structural properties commonly found in semantic parsing into the pre-training language model. To maintain the model's ability to represent real-world data, we also include masked language modeling (MLM) on several existing table-related datasets to regularize our pre-training process. Our proposed pre-training strategy is much data-efficient. When incorporated with strong base semantic parsers, GraPPa achieves new state-of-the-art results on four popular fully supervised and weakly supervised table semantic parsing tasks.
BibTeX
@inproceedings{
yu2021grappa,
title={Gra{\{}PP{\}}a: Grammar-Augmented Pre-Training for Table Semantic Parsing},
author={Tao Yu and Chien-Sheng Wu and Xi Victoria Lin and bailin wang and Yi Chern Tan and Xinyi Yang and Dragomir Radev and richard socher and Caiming Xiong},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=kyaIeYj4zZ}
}