Profit, Fairness, or Both? Setting Priorities in Data Annotation
Workshop paper
Gunay Kazimzade, Milagros Miceli
Abstract
The work of data annotators is fundamental to machine learning and, more broadly, to contemporary knowledge production. This paper summarizes the preliminary results of our investigation into data annotation for vision models. Following a qualitative design, this research project analyzes labeling practices and their possible effects on data and systems, by placing them in the context of market economy. Our results show the prevalence of three market-oriented values embedded in work practices of annotation: profit, standardization, and opacity. Finally, we introduce three elements, namely transparency, education, and regulations, aiming at developing socially responsible annotations, that could help prevent harmful outcomes.
PDF
The work of data annotators is fundamental to machine learning and, more broadly, to contemporary knowledge production. This paper summarizes the preliminary results of our investigation into data annotation for vision models. Following a qualitative design, this research project analyzes labeling practices and their possible effects on data and systems, by placing them in the context of market economy. Our results show the prevalence of three market-oriented values embedded in work practices of annotation: profit, standardization, and opacity. Finally, we introduce three elements, namely transparency, education, and regulations, aiming at developing socially responsible annotations, that could help prevent harmful outcomes.
Cite
APA
Kazimzade, G., & Miceli, M. Profit, Fairness, or Both? Setting Priorities in Data Annotation.
Chicago/Turabian
Kazimzade, Gunay, and Milagros Miceli. Profit, Fairness, or Both? Setting Priorities in Data Annotation, n.d.
MLA
Kazimzade, Gunay, and Milagros Miceli. Profit, Fairness, or Both? Setting Priorities in Data Annotation.