Evaluating Extreme Hierarchical Multi-label Classification

Enrique Amigo; Agustín Delgado

ACL 2022long83 citations

Evaluating Extreme Hierarchical Multi-label Classification

Abstract

Several natural language processing (NLP) tasks are defined as a classification problem in its most complex form: Multi-label Hierarchical Extreme classification, in which items may be associated with multiple classes from a set of thousands of possible classes organized in a hierarchy and with a highly unbalanced distribution both in terms of class frequency and the number of labels per item. We analyze the state of the art of evaluation metrics based on a set of formal properties and we define an information theoretic based metric inspired by the Information Contrast Model (ICM). Experiments on synthetic data and a case study on real data show the suitability of the ICM for such scenarios.

BibTeX

@inproceedings{amigo-delgado-2022-evaluating,
    title = "Evaluating Extreme Hierarchical Multi-label Classification",
    author = "Amigo, Enrique  and
      Delgado, Agust{\'i}n",
    editor = "Muresan, Smaranda  and
      Nakov, Preslav  and
      Villavicencio, Aline",
    booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.acl-long.399/",
    doi = "10.18653/v1/2022.acl-long.399",
    pages = "5809--5819"
}