Nicole Peinelt

GiBERT: Enhancing BERT with Linguistic Information using a Lightweight Gated Injection Method

Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks. However, BERT is only trained to predict missing words – either through masking or next sentence prediction – and has no knowledge of lexical, syntactic or semantic informatio…