Human variation in labeling is often considered noise. Annotation projects for machine
learning (ML) aim at minimizing human label variation, with the assumption to maximize …

Design biases in NLP systems, such as performance differences for different populations,
often stem from their creator's positionality, ie, views and lived experiences shaped by …

Labelled data is the foundation of most natural language processing tasks. However,
labelling data is difficult and there often are diverse valid beliefs about what the correct data …

Abstract Machine learning approaches often require training and evaluation datasets with a
clear separation between positive and negative examples. This requirement overly …

Machine learning models are commonly used to detect toxicity in online conversations.
These models are trained on datasets annotated by human raters. We explore how raters' …

Forming a reliable judgement of a machine learning (ML) model's appropriateness for an
application ecosystem is critical for its responsible use, and requires considering a broad …

NLP datasets annotated with human judgments are rife with disagreements between the
judges. This is especially true for tasks depending on subjective judgments such as …

Legal documents, like contracts or laws, are subject to interpretation. Different people can
have different interpretations of the very same document. Large parts of judicial branches all …

Abstract In Artificial Intelligence research, perspectivism is an approach to machine learning
that aims at leveraging data annotated by different individuals in order to model varied …

In the dynamic landscape of digital forensics, the integration of Artificial Intelligence (AI) and
Machine Learning (ML) stands as a transformative technology, poised to amplify the …