2024
SGLang: Efficient Execution of Structured Language Model Programs
NeurIPS 2024poster
Large language models (LLMs) are increasingly used for complex tasks that require multiple generation calls, advanced prompting techniques, control flow, and structured inputs/outputs. However, efficient systems are lacking for programming and executing these applications. We introduce SGLang, a sys…