Hierarchical Cross-Modality Knowledge Transfer with Sinkhorn Attention for CTC-Based ASR
Due to the modality discrepancy between textual and acoustic modeling, efficiently transferring linguistic knowledge from a pretrained language model (PLM) to acoustic encoding for automatic speech recognition (ASR) still remains a challenging task. In this study, we propose a cross-modality knowled…