2025
Preble: Efficient Distributed Prompt Scheduling for LLM Serving
ICLR 2025poster
Prompts to large language models (LLMs) have evolved beyond simple user questions. For LLMs to solve complex problems, today’s practices are to include domain-specific instructions, illustration of tool usages, and/or long context such as textbook chapters in prompts. As such, many parts of prompts…