Fair Performance Metric Elicitation
Gaurush Hiranandani, Harikrishna Narasimhan, Sanmi Koyejo
Abstract
What is a fair performance metric? We consider the choice of fairness metrics through the lens of metric elicitation -- a principled framework for selecting performance metrics that best reflect implicit preferences. The use of metric elicitation enables a practitioner to tune the performance and fairness metrics to the task, context, and population at hand. Specifically, we propose a novel strategy to elicit group-fair performance metrics for multiclass classification problems with multiple sensitive groups that also includes selecting the trade-off between predictive performance and fairness violation. The proposed elicitation strategy requires only relative preference feedback and is robust to both finite sample and feedback noise.
BibTeX
@inproceedings{NEURIPS2020_7ec2442a,
author = {Hiranandani, Gaurush and Narasimhan, Harikrishna and Koyejo, Sanmi},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {11083--11095},
publisher = {Curran Associates, Inc.},
title = {Fair Performance Metric Elicitation},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/7ec2442aa04c157590b2fa1a7d093a33-Paper.pdf},
volume = {33},
year = {2020}
}