Santosh Kumar Yadav

SLBERT: A Novel Pre-Training Framework for Joint Speech and Language Modeling

We propose SLBERT (Speech and Language pre-training framework for BERT), an end-to-end trainable framework for learning joint representations of speech and language modalities. We enhance the well-known BERT architecture to provide a dual-stream multimodal architecture that processes both speech and…