AIConfPaper
← Search

Wael Hamza

13 accepted papers

2024

Towards ASR Robust Spoken Language Understanding Through in-Context Learning with Word Confusion Networks

Kevin Everson, Yile Gu, Chao-Han Huck Yang, Prashanth Gurunath Shivakumar, Guan-Ting Lin, Jari Kolehmainen +7
ICASSP 2024accepted

In the realm of spoken language understanding (SLU). numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with transcribed speech instead of conventional written text. In real-world scenarios, prior to input into an LLM. an automated…

Cited by 0Source ↗Scholar ↗
2023

Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models

Saleh Soltan, Andy Rosenbaum, Tobias Falke, Qin Lu, Anna Rumshisky, Wael Hamza
ACL 2023findings

Pre-trained encoder-only and sequence-to-sequence (seq2seq) models each have advantages, however training both model types from scratch is computationally expensive. We explore recipes to improve pre-training efficiency by initializing one model from the other. (1) Extracting the encoder from a seq2…

Cited by 0Source ↗PDF ↗code ↗Scholar ↗
2022

A Hybrid Approach to Cross-lingual Product Review Summarization

Saleh Soltan, Victor Soto, Ke Tran, Wael Hamza
EMNLP 2022industry

We present a hybrid approach for product review summarization which consists of: (i) an unsupervised extractive step to extract the most important sentences out of all the reviews, and (ii) a supervised abstractive step to summarize the extracted sentences into a coherent short summary. This approac…

Cited by 0Source ↗PDF ↗Scholar ↗
2022

Attention Fusion: a light yet efficient late fusion mechanism for task adaptation in NLU

Jin Cao, Chandana Satya Prakash, Wael Hamza
NAACL 2022findings

Fine-tuning a pre-trained language model using annotated data has become the de-facto standard for adapting general-purpose pre-trained models like BERT to downstream tasks. However, given the trend of larger pre-trained models, fine-tuning these models for each downstream task is parameter-ineffici…

Cited by 11Source ↗PDF ↗Scholar ↗
2022

Controlled Data Generation via Insertion Operations for NLU

Manoj Kumar, Yuval Merhav, Haidar Khan, Rahul Gupta, Anna Rumshisky, Wael Hamza
NAACL 2022industry

Use of synthetic data is rapidly emerging as a realistic alternative to manually annotating live traffic for industry-scale model building. Manual data annotation is slow, expensive and not preferred for meeting customer privacy expectations. Further, commercial natural language applications are req…

Cited by 6Source ↗PDF ↗Scholar ↗
2022

Instilling Type Knowledge in Language Models via Multi-Task QA

Shuyang Li, Mukund Sridhar, Chandana Satya Prakash, Jin Cao, Wael Hamza, Julian McAuley
NAACL 2022findings

Understanding human language often necessitates understanding entities and their place in a taxonomy of knowledge—their types.Previous methods to learn entity types rely on training classifiers on datasets with coarse, noisy, and incomplete labels. We introduce a method to instill fine-grained type…

Cited by 10Source ↗PDF ↗code ↗Scholar ↗
2022

LINGUIST: Language Model Instruction Tuning to Generate Annotated Utterances for Intent Classification and Slot Tagging

Andy Rosenbaum, Saleh Soltan, Wael Hamza, Yannick Versley, Markus Boese
COLING 2022main

We present LINGUIST, a method for generating annotated data for Intent Classification and Slot Tagging (IC+ST), via fine-tuning AlexaTM 5B, a 5-billion-parameter multilingual sequence-to-sequence (seq2seq) model, on a flexible instruction prompt. In a 10-shot novel intent setting for the SNIPS datas…

Cited by 36Source ↗PDF ↗code ↗Scholar ↗
2022

Training Naturalized Semantic Parsers with Very Little Data

Subendhu Rongali, Konstantine Arkoudas, Melanie Rubino, Wael Hamza
IJCAI 2022poster

Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectures based on large language models that have been pretrained on vast amounts of text. To better leverage that pretraining…

Cited by 15Source ↗PDF ↗code ↗Scholar ↗
2021

Contextual Domain Classification with Temporal Representations

Tzu-Hsiang Lin, Yipeng Shi, Chentao Ye, Yang Fan, Weitong Ruan, Emre Barut +2
NAACL 2021industry

In commercial dialogue systems, the Spoken Language Understanding (SLU) component tends to have numerous domains thus context is needed to help resolve ambiguities. Previous works that incorporate context for SLU have mostly focused on domains where context is limited to a few minutes. However, ther…

Cited by 1Source ↗PDF ↗Scholar ↗
2021

Exploring Transfer Learning For End-to-End Spoken Language Understanding

Subendhu Rongali, Beiye Liu, Liwei Cai, Konstantine Arkoudas, Chengwei Su, Wael Hamza
AAAI 2021technical

Voice Assistants such as Alexa, Siri, and Google Assistant typically use a two-stage Spoken Language Understanding pipeline; first, an Automatic Speech Recognition (ASR) component to process customer speech and generate text transcriptions, followed by a Natural Language Understanding (NLU) componen…

Cited by 28Source ↗PDF ↗code ↗Scholar ↗
2020

Multi-task Learning of Spoken Language Understanding by Integrating N-Best Hypotheses with Hierarchical Attention

Mingda Li, Xinyue Liu, Weitong Ruan, Luca Soldaini, Wael Hamza, Chengwei Su
COLING 2020industry

Currently, in spoken language understanding (SLU) systems, the automatic speech recognition (ASR) module produces multiple interpretations (or hypotheses) for the input audio signal and the natural language understanding (NLU) module takes the one with the highest confidence score for domain or inte…

Cited by 8Source ↗PDF ↗Scholar ↗