Automatic broadcast news summarization via rank classifiers and crowdsourced annotation

Abstract

Extractive speech summarization methods generally operate as a binary classifier deciding if a sentence belongs to the summary or not. However, it is well known that even human annotators do not agree on selecting most summary sentences. In this paper, we take a probabilistic view of the summarization ground-truth and assume that more frequently selected sentences by annotators are of higher importance. Using a large summary data-set obtained through crowdsourcing, we empirically show that sentence selection frequency is inversely related to its summarization rank. Consequently, we model the relative importance between sentences using a rank-based classifier. Additionally, we utilize an extended paralinguistic feature set that has not been previously used for speech summarization. Lexical and structural features are also included. Support Vector Machine (SVM) is used as the baseline binary classifier and rank classifier. Experimental evaluations show that the proposed approach outperforms traditional binary classifiers with respect to various ROUGE summarization metrics for different summarization compression ratios (CR).