Xizheng Wang

Transcending Cost-Quality Tradeoff in Agent Serving via Session-Awareness

Large Language Model (LLM) agents are capable of task execution across various domains by autonomously interacting with environments and refining LLM responses based on feedback. However, existing model serving systems are not optimized for the unique demands of serving agents. Compared to classic m…