Yuanyuan Chloe Yang

T-TAMER: Provably Taming Trade-offs in ML Serving

As machine learning models continue to grow in size and complexity, efficient serving faces increasingly broad trade-offs spanning accuracy, latency, resource usage, and other objectives. Multi-model serving further complicates these trade-offs; for example, in cascaded models, each early-exit decis…