Sébastien Melo

Epistemic Uncertainty Quantification To Improve Decisions From Black-Box Models

Distinguishing a model's lack of knowledge (epistemic uncertainty) from inherent task randomness (aleatoric uncertainty) is crucial for reliable AI. However, standard evaluation metrics of confidence scores target different aspects. AUC and accuracy capture predictive signal, proper scoring rules ca…