Conformal Prediction for Partial Label Learning
Xiuwen Gong, Nitin Bisht, Guandong Xu
Abstract
Partial label learning (PLL) allows each instance to be annotated with a set of candidate labels, but only one is the ground-truth label. Although the state-of-the-art (SOTA) PLL models have shown competitive performance, they cannot get rid of the negative influence from the noisy false-positive labels during the training process. This leads to a large extent of uncertainty of PLL models’ prediction, and it becomes unreliable to trust a PLL model’s performance only by its prediction accuracy. To bridge this gap, we develop a new framework to quantify the uncertainty for PLL models with valid confidence guarantee, which is named as Conformal Prediction for Partial Label Learning (CP-PLL). This framework can be implemented on top of any PLL method to quantify their predictive confidence in terms of average prediction set size with a use-specified error rate or coverage/confidence level (i.e., probability). We prove that the coverage guarantee in PLL still holds, that is, the ground-truth label can be covered in the constructed prediction set with the user pre-defined error rate α when we use the noisy calibration data to carlibrate the PLL models, which yields to a probability interval of [1- α, 1- α + 1/n+1 + ε]. Extensive experiments are conducted on SOTA PLL methods and benchmark datasets to verify the effectiveness of the proposed framework.
BibTeX
@article{Gong_Bisht_Xu_2025, title={Conformal Prediction for Partial Label Learning}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/33853}, DOI={10.1609/aaai.v39i16.33853}, abstractNote={Partial label learning (PLL) allows each instance to be annotated with a set of candidate labels, but only one is the ground-truth label. Although the state-of-the-art (SOTA) PLL models have shown competitive performance, they cannot get rid of the negative influence from the noisy false-positive labels during the training process. This leads to a large extent of uncertainty of PLL models’ prediction, and it becomes unreliable to trust a PLL model’s performance only by its prediction accuracy. To bridge this gap, we develop a new framework to quantify the uncertainty for PLL models with valid confidence guarantee, which is named as Conformal Prediction for Partial Label Learning (CP-PLL). This framework can be implemented on top of any PLL method to quantify their predictive confidence in terms of average prediction set size with a use-specified error rate or coverage/confidence level (i.e., probability). We prove that the coverage guarantee in PLL still holds, that is, the ground-truth label can be covered in the constructed prediction set with the user pre-defined error rate α when we use the noisy calibration data to carlibrate the PLL models, which yields to a probability interval of [1- α, 1- α + 1/n+1 + ε]. Extensive experiments are conducted on SOTA PLL methods and benchmark datasets to verify the effectiveness of the proposed framework.}, number={16}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Gong, Xiuwen and Bisht, Nitin and Xu, Guandong}, year={2025}, month={Apr.}, pages={16862-16870} }