Jaeeun Kil

ICaRus: Identical Cache Reuse for Efficient Multi-Model Inference

Multi model inference, where multiple task-specialized models collaborate to solve complex real-world problems, has recently emerged as a prominent paradigm, particularly in the development of agentic AI systems. However, in such scenarios, each model must maintain its own Key-Value (KV) cache for t…