Evaluating Snippet Significance: A Framework for Audio and Text-Based Dialogue Summarization

Abstract

The extraction and summarization of key information from dialogues are essential for improving communication tools and outcomes across various domains. Traditional text-based approaches often fail to capture critical auditory cues such as intonation, pauses, and emphasis, which are vital for understanding the full context of spoken interactions. This paper introduces a novel framework that independently analyzes text and audio modalities, assigning significance scores to dialogue snippets. Evaluations on the AMI [1] and ICSI [2] meeting corpora utilize Wav2Vec 2.0 [3] for audio embedding extraction and BERT for text embeddings, allowing the contributions of each modality to be thoroughly examined. The method demonstrates a significant improvement in summarization accuracy over text-only models. On the AMI [1] dataset, the proposed approach achieves ROUGE-1 (0.94), ROUGE-2 (0.93), ROUGE-L (0.94), and BLEU (0.88), outperforming state-of-the-art models such as BART and PEGASUS. Similarly, on the ICSI [2] dataset, it achieves ROUGE-1 (0.79), ROUGE-2 (0.75), and BLEU (0.47), showcasing its robustness across datasets. Furthermore, audio-based summarization achieves higher significance scores (94%) compared to text-based (89%), highlighting the additional context provided by auditory features. The findings indicate that audio features are crucial for capturing emotional tone, speaker intent, and conversational flow, which are often overlooked by text-based methods. By integrating both modalities, this framework produces more contextually rich and accurate summaries. This research sets a new benchmark for dialogue summarization by demonstrating that incorporating auditory cues leads to a deeper and more comprehensive understanding of dialogues, with applications ranging from virtual meetings to educational content summarization.