Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention
Pengcheng Yin, Hao Fang, Graham Neubig, Adam Pauls, Emmanouil Antonios Platanios, Yu Su, Sam Thomson, Jacob Andreas
Abstract
We describe a span-level supervised attention loss that improves compositional generalization in semantic parsers. Our approach builds on existing losses that encourage attention maps in neural sequence-to-sequence models to imitate the output of classical word alignment algorithms. Where past work has used word-level alignments, we focus on spans; borrowing ideas from phrase-based machine translation, we align subtrees in semantic parses to spans of input sentences, and encourage neural attention mechanisms to mimic these alignments. This method improves the performance of transformers, RNNs, and structured decoders on three benchmarks of compositional generalization.
BibTeX
@inproceedings{yin-etal-2021-compositional,
title = "Compositional Generalization for Neural Semantic Parsing via Span-level Supervised Attention",
author = "Yin, Pengcheng and
Fang, Hao and
Neubig, Graham and
Pauls, Adam and
Platanios, Emmanouil Antonios and
Su, Yu and
Thomson, Sam and
Andreas, Jacob",
editor = "Toutanova, Kristina and
Rumshisky, Anna and
Zettlemoyer, Luke and
Hakkani-Tur, Dilek and
Beltagy, Iz and
Bethard, Steven and
Cotterell, Ryan and
Chakraborty, Tanmoy and
Zhou, Yichao",
booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
month = jun,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.naacl-main.225/",
doi = "10.18653/v1/2021.naacl-main.225",
pages = "2810--2823"
}