Seunggeun Cho

SpecEdge: Scalable Edge-Assisted Serving Framework for Interactive LLMs

Large language models (LLMs) power many modern applications, but serving them at scale remains costly and resource-intensive. Current server-centric systems overlook consumer-grade GPUs at the edge. We introduce SpecEdge, an edge-assisted inference framework that splits LLM workloads between edge an…