2025
Efficient Inter-Operator Scheduling for Concurrent Recommendation Model Inference on GPU
IJCAI 2025
Deep learning-based recommendation systems are increasingly important in the industry. To meet strict SLA requirements, serving frameworks must efficiently handle concurrent queries. However, current serving systems fail to serve concurrent queries due to the following problems: (1) inefficient oper