kNN-CM: A Non-parametric Inference-Phase Adaptation of Parametric Text Classifiers
Rishabh Bhardwaj, Yingting Li, Navonil Majumder, Bo Cheng, Soujanya Poria
Abstract
Semi-parametric models exhibit the properties of both parametric and non-parametric modeling and have been shown to be effective in the next-word prediction language modeling task. However, there is a lack of studies on the text-discriminating properties of such models. We propose an inference-phase approach---\textit{k}-Nearest Neighbor Classification Model (\textit{k}NN-CM)---that enhances the capacity of a pre-trained parametric text classifier by incorporating a simple neighborhood search through the representation space of (memorized) training samples. The final class prediction of \textit{k}NN-CM is based on the convex combination of probabilities obtained from \textit{k}NN search and prediction of the classifier. Our experiments show consistent performance improvements on eight SuperGLUE tasks, three adversarial natural language inference (ANLI) datasets, 11 question-answering (QA) datasets, and two sentiment classification datasets.
BibTeX
@inproceedings{
bhardwaj2023knncm,
title={k{NN}-{CM}: A Non-parametric Inference-Phase Adaptation of Parametric Text Classifiers},
author={Rishabh Bhardwaj and Yingting Li and Navonil Majumder and Bo Cheng and Soujanya Poria},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://openreview.net/forum?id=9BuTdxSfIO}
}