2025
DiSCo: Device-Server Collaborative LLM-based Text Streaming Services
ACL 2025finding
The rapid rise of large language models (LLMs) in text streaming services has introduced significant cost and Quality of Experience (QoE) challenges in serving millions of daily requests, especially in meeting Time-To-First-Token (TTFT) and Time-Between-Token (TBT) requirements for real-time interac…