AAAI 2024technical1 citations
Meta-Crafting: Improved Detection of Out-of-Distributed Texts via Crafting Metadata Space (Student Abstract)
Ryan Koo, Yekyung Kim, Dongyeop Kang, Jaehyung Kim
Abstract
Detecting out-of-distribution (OOD) samples is crucial for robust NLP models. Recent works observe two OOD types: background shifts (style change) and semantic shifts (content change), but existing detection methods vary in effectiveness for each type. To this end, we propose Meta-Crafting, a unified OOD detection method by constructing a new discriminative feature space utilizing 7 model-driven metadata chosen empirically that well detects both types of shifts. Our experimental results demonstrate state-of-the-art robustness to both shifts and significantly improved detection on stress datasets.
BibTeX
@article{Koo_Kim_Kang_Kim_2024, title={Meta-Crafting: Improved Detection of Out-of-Distributed Texts via Crafting Metadata Space (Student Abstract)}, volume={38}, url={https://ojs.aaai.org/index.php/AAAI/article/view/30467}, DOI={10.1609/aaai.v38i21.30467}, abstractNote={Detecting out-of-distribution (OOD) samples is crucial for robust NLP models. Recent works observe two OOD types: background shifts (style change) and semantic shifts (content change), but existing detection methods vary in effectiveness for each type. To this end, we propose Meta-Crafting, a unified OOD detection method by constructing a new discriminative feature space utilizing 7 model-driven metadata chosen empirically that well detects both types of shifts. Our experimental results demonstrate state-of-the-art robustness to both shifts and significantly improved detection on stress datasets.}, number={21}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Koo, Ryan and Kim, Yekyung and Kang, Dongyeop and Kim, Jaehyung}, year={2024}, month={Mar.}, pages={23548-23549} }