2026
When RL Meets Adaptive Speculative Training: A Unified Training-Serving System
ICML 2026poster
Speculative decoding can significantly accelerate LLM serving, but its real-world benefits often erode due to training–serving mismatch and non-stationary traffic. Unlike previous systems that decouple speculator training from inference, we present a unified training–serving system, Aurora, that clo…