Shruti Palaskar — accepted AI-conference papers

2026

VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety

Shruti Palaskar, Leon Alexander Gatys, Mona Abdelrahman, Mar Jacobo, Laurence F Lindsey, Rutika Moharir +6

ICLR 2026poster

Safety evaluation of multimodal foundation models often treats vision and language inputs separately, missing risks from joint interpretation where benign content becomes harmful in combination. Existing approaches also fail to distinguish clearly unsafe content from borderline cases, leading to pro…

Cited by 0 Source ↗code ↗Scholar ↗

2022

End-to-End Speech Summarization Using Restricted Self-Attention

Roshan Sharma, Shruti Palaskar, Alan W. Black, Florian Metze

ICASSP 2022accepted

Speech summarization is typically performed by using a cascade of speech recognition and text summarization models. End-to-end modeling of speech summarization models is challenging due to memory and compute constraints arising from long input audio sequences. Recent work in document summarization h…

Cited by 0 Source ↗Scholar ↗

2022

On Advances in Text Generation from Images Beyond Captioning: A Case Study in Self-Rationalization

Shruti Palaskar, Akshita Bhagia, Yonatan Bisk, Florian Metze, Alan W Black, Ana Marasovic

EMNLP 2022finding

Combining the visual modality with pretrained language models has been surprisingly effective for simple descriptive tasks such as image captioning. More general text generation however remains elusive. We take a step back and ask: How do these models work for more complex generative tasks, i.e. con…

Cited by 3 Source ↗PDF ↗Scholar ↗

2021

How2Sign: A Large-Scale Multimodal Dataset for Continuous American Sign Language

Amanda Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze +2

CVPR 2021poster

One of the factors that have hindered progress in the areas of sign language recognition, translation, and production is the absence of large annotated datasets. Towards this end, we introduce How2Sign, a multimodal and multiview continuous American Sign Language (ASL) dataset, consisting of a paral…

Cited by 257 PDF ↗code ↗Scholar ↗

2020

ASR Error Correction and Domain Adaptation Using Machine Translation

Anirudh Mani, Shruti Palaskar, Nimshi Venkat Meripo, Sandeep Konam, Florian Metze

ICASSP 2020accepted

Off-the-shelf pre-trained Automatic Speech Recognition (ASR) systems are an increasingly viable service for companies of any size building speech-based products. While these ASR systems are trained on large amounts of data, domain mismatch is still an issue for many such parties that want to use thi…

Cited by 0 Source ↗Scholar ↗

2019

Learned in Speech Recognition: Contextual Acoustic Word Embeddings

Shruti Palaskar, Vikas Raunak, Florian Metze

ICASSP 2019accepted

End-to-end acoustic-to-word speech recognition models have recently gained popularity because they are easy to train, scale well to large amounts of training data, and do not require a lexicon. In addition, word models may also be easier to integrate with downstream tasks such as spoken language und…

Cited by 0 Source ↗Scholar ↗

2019

Learning from Multiview Correlations in Open-domain Videos

Nils Holzenberger, Shruti Palaskar, Pranava Madhyastha, Florian Metze, Raman Arora

ICASSP 2019accepted

An increasing number of datasets contain multiple views, such as video, sound and automatic captions. A basic challenge in representation learning is how to leverage multiple views to learn better representations. This is further complicated by the existence of a latent alignment between views, such…

Cited by 0 Source ↗Scholar ↗

2019

Multimodal Grounding for Sequence-to-sequence Speech Recognition

Ozan Caglayan, Ramon Sanabria, Shruti Palaskar, Loïc Barrault, Florian Metze

ICASSP 2019accepted

Humans are capable of processing speech by making use of multiple sensory modalities. For example, the environment where a conversation takes place generally provides semantic and/or acoustic context that helps us to resolve ambiguities or to recall named entities. Motivated by this, there have been…

Cited by 0 Source ↗Scholar ↗

2018

End-to-end Multimodal Speech Recognition

Shruti Palaskar, Ramon Sanabria, Florian Metze

ICASSP 2018accepted

Transcription or sub-titling of open-domain videos is still a challenging domain for Automatic Speech Recognition (ASR) due to the data's challenging acoustics, variable signal processing and the essentially unrestricted domain of the data. In previous work, we have shown that the visual channel - s…

Cited by 0 Source ↗Scholar ↗

2018

Linguistic Unit Discovery from Multi-Modal Inputs in Unwritten Languages: Summary of the "Speaking Rosetta" JSALT 2017 Workshop

Odette Scharenborg, Laurent Besacier, Alan W. Black, Mark Hasegawa-Johnson, Florian Metze, Graham Neubig +13

ICASSP 2018accepted

We summarize the accomplishments of a multi-disciplinary workshop exploring the computational and scientific issues surrounding the discovery of linguistic units (subwords and words) in a language without orthography. We study the replacement of orthographic transcriptions by images and/or translate…

Cited by 0 Source ↗Scholar ↗