← Search

Nathan Schucher

2 accepted papers

2021

PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models

EMNLP 2021main

Large pre-trained language models for textual data have an unconstrained output space; at each decoding step, they can produce any of 10,000s of sub-word tokens. When fine-tuned to target constrained formal languages like SQL, these models often generate invalid code, rendering it unusable. We propo…