2025
PARALLELPROMPT: Extracting Parallelism from Large Language Model Queries
NeurIPS 2025poster
LLM serving systems typically treat user prompts as monolithic inputs, optimizing inference through decoding tricks or inter-query batching. However, many real-world prompts contain *latent semantic parallelism*—decomposable structures where subtasks can be executed independently to reduce latency w…