2025
Demystifying Cost-Efficiency in LLM Serving over Heterogeneous GPUs
ICML 2025poster
Recent advancements in Large Language Models (LLMs) have led to increasingly diverse requests, accompanied with varying resource (compute and memory) demands to serve them. However, this in turn degrades the cost-efficiency of LLM serving as common practices primarily rely on homogeneous GPU resourc…