Towards Zero-shot Cross-lingual SLU with Syntax-aware Multi-view Contrastive Learning

Abstract

Recent state-of-the-art zero-shot cross-lingual spoken language understanding (SLU) models utilize contrastive learning to achieve multilingual semantics alignment between the original utterance and code-switched counterpart. Despite achieving promising results, we discover that they still suffer from two main issues: the neglect of incorporating language-agnostic information into language models and the lack of supervision for extracting fruitful semantics from unpaired samples. To tackle the two issues, we propose Syntax-aware Multi-view Contrastive Learning termed SMCL for zero-shot cross-lingual SLU. In detail, to solve the first issue, we propose a syntax-aware module that incorporates dependency relations and POS tags into multi-lingual language models, thus explicitly leveraging transferable syntax information across different languages. To solve the second issue, we propose multi-view contrastive learning to achieve semantic consistency within and among pairs. Extensive experiments on the benchmark across nine languages show that our SMCL achieves new state-of-the-art performances.