Unsupervised and Few-Shot Parsing from Pretrained Language Models (Extended Abstract)

Zhiyuan Zeng; Deyi Xiong

IJCAI 2023poster0 citations

Unsupervised and Few-Shot Parsing from Pretrained Language Models (Extended Abstract)

Abstract

This paper proposes two Unsupervised constituent Parsing models (UPOA and UPIO) that calculate inside and outside association scores solely based on the self-attention weight matrix learned in a pretrained language model. The proposed unsupervised parsing models are further extended to few-shot parsing models (FPOA, FPIO) that use a few annotated trees to fine-tune the linear projection matrices in self-attention. Experiments on PTB and SPRML show that both unsupervised and few-shot parsing methods are better than or comparable to the previous methods.

Natural Language Processing: NLP: Tagging, chunking, and parsingNatural Language Processing: General

BibTeX

@inproceedings{ijcai2023p797,
  title     = {Unsupervised and Few-Shot Parsing from Pretrained Language Models (Extended Abstract)},
  author    = {Zeng, Zhiyuan and Xiong, Deyi},
  booktitle = {Proceedings of the Thirty-Second International Joint Conference on
               Artificial Intelligence, {IJCAI-23}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Edith Elkind},
  pages     = {6995--7000},
  year      = {2023},
  month     = {8},
  note      = {Journal Track},
  doi       = {10.24963/ijcai.2023/797},
  url       = {https://doi.org/10.24963/ijcai.2023/797},
}