2026
Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatch
ICML 2026poster
Deterministic inference is increasingly critical for large language model (LLM) applications such as LLM-as-a-judge evaluation, multi-agent systems, and Reinforcement Learning (RL). However, existing LLM serving frameworks exhibit non-deterministic behavior: identical inputs can yield different outp…