AI tools are increasingly deployed in community contexts. However, datasets used to
evaluate AI are typically created by developers and annotators outside a given community …

Attempting to make sense of a phenomenon or crisis, social media users often share data
visualizations and interpretations that can be erroneous or misleading. Prior work has …

In this work, we take a first step towards designing summarization systems that are faithful to
the author's opinions and perspectives. Focusing on a case study of preserving political …

Line managers form the first level of management in organizations, and must make complex
decisions, while maintaining relationships with those impacted by their decisions. Amidst …

There has been growing interest from both practitioners and researchers in engaging end
users in AI auditing, to draw upon users' unique knowledge and lived experiences …

Large language model (LLM) evaluations often assume there is a single correct response--a
gold label--for each item in the evaluation corpus. However, some tasks can be ambiguous …

Community and organizational policies are typically designed in a top-down, centralized
fashion, with limited input from impacted stakeholders. This can result in policies that are …

The New York Times Connections game has emerged as a popular and challenging pursuit
for word puzzle enthusiasts. We collect 200 Connections games to evaluate the …

The rapid growth of submissions to top-tier Artificial Intelligence (AI) and Machine Learning
(ML) conferences has prompted many venues to transition from closed to open review …

Annotating texts can be a tedious task, especially when texts are noisy. At the root of the
issue, guidelines are not always optimized enough to be able to perform the required …