← Search

Ahmed Aly

5 accepted papers

2024

LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding

ACL 2024long

We present LayerSkip, an end-to-end solution to speed-up inference of large language models (LLMs). First, during training we apply layer dropout, with low dropout rates for earlier layers and higher dropout rates for later layers, and an early exit loss where all transformer layers share the same e…

2023

ICASSP 2023 Spoken Language Understanding Grand Challenge

ICASSP 2023accepted

Spoken language understanding (SLU) is a important field between the Speech and NLP community focused on converting a users’ speech utterance into an executable semantic parse. In order to facilitate open research in this space, we introduce the 1st Spoken Language Understanding challenge hosted at…

Cited by 0SourceScholar
2021

Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog

NAACL 2021long

Semantic parsing using sequence-to-sequence models allows parsing of deeper representations compared to traditional word tagging based models. In spite of these advantages, widespread adoption of these models for real-time conversational use cases has been stymied by higher compute requirements and…

2021

Span Pointer Networks for Non-Autoregressive Task-Oriented Semantic Parsing

EMNLP 2021finding

An effective recipe for building seq2seq, non-autoregressive, task-oriented parsers to map utterances to semantic frames proceeds in three steps: encoding an utterance x, predicting a frame’s length |y|, and decoding a |y|-sized frame with utterance and ontology tokens. Though empirically strong, th…

Cited by 25SourcePDFScholar