Crowdworksheets: Accounting for individual and collective identities underlying crowdsourced dataset annotation
Human annotated data plays a crucial role in machine learning (ML) research and
development. However, the ethical considerations around the processes and decisions that …
development. However, the ethical considerations around the processes and decisions that …
Hate speech classifiers learn normative social stereotypes
Social stereotypes negatively impact individuals' judgments about different groups and may
have a critical role in understanding language directed toward marginalized groups. Here …
have a critical role in understanding language directed toward marginalized groups. Here …
Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation
Machine learning models are commonly used to detect toxicity in online conversations.
These models are trained on datasets annotated by human raters. We explore how raters' …
These models are trained on datasets annotated by human raters. We explore how raters' …
The PRISM alignment dataset: What participatory, representative and individualised human feedback reveals about the subjective and multicultural alignment of large …
Human feedback is central to the alignment of Large Language Models (LLMs). However,
open questions remain about the methods (how), domains (where), people (who) and …
open questions remain about the methods (how), domains (where), people (who) and …
Toxicity detection: Does context really matter?
Moderation is crucial to promoting healthy on-line discussions. Although severaltoxicity'
detection datasets and models have been published, most of them ignore the context of the …
detection datasets and models have been published, most of them ignore the context of the …
Commonality and variation in mental representations of music revealed by a cross-cultural comparison of rhythm priors in 15 countries
Music is present in every known society but varies from place to place. What, if anything, is
universal to music cognition? We measured a signature of mental representations of rhythm …
universal to music cognition? We measured a signature of mental representations of rhythm …
Finding microaggressions in the wild: A case for locating elusive phenomena in social media posts
Microaggressions are subtle, often veiled, manifestations of human biases. These uncivil
interactions can have a powerful negative impact on people by marginalizing minorities and …
interactions can have a powerful negative impact on people by marginalizing minorities and …
Whose ground truth? accounting for individual and collective identities underlying dataset annotation
Human annotations play a crucial role in machine learning (ML) research and development.
However, the ethical considerations around the processes and decisions that go into …
However, the ethical considerations around the processes and decisions that go into …
Convenience samples in political science experiments
This chapter provides an overview of convenience sampling in political science
experimental research. We first define convenience sampling and track its use in political …
experimental research. We first define convenience sampling and track its use in political …
Frustration: still a common user experience
When computers unexpectedly delay or thwart goal attainment, frustration ensues. The
central studies of the extent, content, and impact of such frustration were done more than 15 …
central studies of the extent, content, and impact of such frustration were done more than 15 …