SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token

IKHYUN CHO; Yoonhwa Jung; Julia Hockenmaier

EMNLP 2023long findings0 citations

SIR-ABSC: Incorporating Syntax into RoBERTa-based Sentiment Analysis Models with a Special Aggregator Token

IKHYUN CHO, Yoonhwa Jung, Julia Hockenmaier

Abstract

We present a simple, but effective method to incorporate syntactic dependency information directly into transformer-based language models (e.g. RoBERTa) for tasks such as Aspect-Based Sentiment Classification (ABSC), where the desired output depends on specific input tokens. In contrast to prior approaches to ABSC that capture syntax by combining language models with graph neural networks over dependency trees, our model, Syntax-Integrated RoBERTa for ABSC (SIR-ABSC) incorporates syntax directly into the language model by using a novel aggregator token. Yet, SIR-ABSC outperforms these more complex models, yielding new state-of-the-art results on ABSC.

Aspect-based sentiment analysisPre-trained Language ModelsRoBERTa

BibTeX

@inproceedings{
cho2023sirabsc,
title={{SIR}-{ABSC}: Incorporating Syntax into Ro{BERT}a-based Sentiment Analysis Models with a Special Aggregator Token},
author={IKHYUN CHO and Yoonhwa Jung and Julia Hockenmaier},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://openreview.net/forum?id=T9wuVnNa5v}
}