Hannah Scheufele

Quantifying Temperature Scaling in Discrete Sequence (Language) Models

Naive application of token-wise temperature scaling alters the maximum a posteriori (MAP) estimate at the sequence level, degrading model performance. This issue is exacerbated in MDMs, which estimate sequence-level likelihoods with high variance under different unmasking orders. In this paper, we a…