2022
DeepStruct: Pretraining of Language Models for Structure Prediction
ACL 2022findings
We introduce a method for improving the structural understanding abilities of language models. Unlike previous approaches that finetune the models with task-specific augmentation, we pretrain language models to generate structures from the text on a collection of task-agnostic corpora. Our structure…