The increase of the amount of misinformation spread every day online is a huge threat to the
society. Organizations and researchers are working to contrast this misinformation plague. In …

This paper does not describe a novel method. Instead, it studies an essential foundation for
reliable benchmarking and ultimately real-world application of AI-based image analysis …

When groups of people are tasked with making a judgment, the issue of uncertainty often
arises. Existing methods to reduce uncertainty typically focus on iteratively improving …

Crowdsourcing platforms have transformed distributed problem-solving, yet quality control
remains a persistent challenge. Traditional quality control measures, such as prescreening …

Human annotations are vital to supervised learning, yet annotators often disagree on the
correct label, especially as annotation tasks increase in complexity. A common strategy to …

Human face images represent a rich set of visual information for online social media
platforms to optimize the machine learning (ML)/AI models in their data-driven facial …

It is common practice for machine learning systems to rely on crowdsourced label data for
training and evaluation. It is also well-known that biases present in the label data can induce …

Misinformation has been rapidly spreading online. The common approach to dealing with it
is deploying expert fact-checkers who follow forensic processes to identify the veracity of …

Crowdsourcing markets are expanding worldwide, but often feature standardized interfaces
that ignore the cultural diversity of their workers, negatively impacting their well-being and …

Ancient calligraphy manuscripts often suffered from degradation due to human factors and
natural weathering. Image restoration techniques restore the degraded contents in …