ITMO language diarization and identification systems for the DISPLACE 2024 challenge

Abstract

This paper describes our language diarization and identification systems developed for far-field recorded group conversations. Our approach has a two-stage design and relies on classical methods, such as spectral clustering of language embeddings. The heuristic bypass (HBP) method was utilized to generate the similarity matrix required for spectral clustering used in the first stage. In the second stage the language identification block predicts language labels for a specific set of target languages. Users can manually determine the number of clusters for spectral clustering when using the identification block into the processing pipeline. Various language embedding extractors, including those based on ResNet34 and wav2vec 2.0 architectures, were utilized. We used these systems, as well as their fusion, into submission for Track 2 on language diarization of the DISPLACE 2024 challenge. Our system achieved 5 % relative improvements on eval set compared to the organizer-provided baseline system, securing the second place for Track 2 of the challenge.