A checklist to combat cognitive biases in crowdsourcing

T Draws, A Rieger, O Inel, U Gadiraju… - Proceedings of the AAAI …, 2021 - ojs.aaai.org
Recent research has demonstrated that cognitive biases such as the confirmation bias or the
anchoring effect can negatively affect the quality of crowdsourced data. In practice, however …

Moral machine or tyranny of the majority?

M Feffer, H Heidari, ZC Lipton - Proceedings of the AAAI Conference on …, 2023 - ojs.aaai.org
With artificial intelligence systems increasingly applied in consequential domains,
researchers have begun to ask how AI systems ought to act in ethically charged situations …

A new binary programming formulation and social choice property for Kemeny rank aggregation

Y Yoo, AR Escobedo - Decision Analysis, 2021 - pubsonline.informs.org
Rank aggregation is widely used in group decision making and many other applications,
where it is of interest to consolidate heterogeneous ordered lists. Oftentimes, these rankings …

An axiomatic distance methodology for aggregating multimodal evaluations

AR Escobedo, E Moreno-Centeno, R Yasmin - Information Sciences, 2022 - Elsevier
This work introduces a multimodal data aggregation methodology featuring optimization
models and algorithms for jointly aggregating heterogeneous ordinal and cardinal …

Improving crowdsourcing-based image classification through expanded input elicitation and machine learning

R Yasmin, MM Hassan, JT Grassel… - Frontiers in Artificial …, 2022 - frontiersin.org
This work investigates how different forms of input elicitation obtained from crowdsourcing
can be utilized to improve the quality of inferred labels for image classification tasks, where …

Relaxed forced choice improves performance of visual quality assessment methods

M Jenadeleh, J Zagermann, H Reiterer… - … on Quality of …, 2023 - ieeexplore.ieee.org
In image quality assessment, a collective visual quality score for an image or video is
obtained from the individual ratings of many subjects. One commonly used format for these …

Elicitation and aggregation of multimodal estimates improve wisdom of crowd effects on ordering tasks

Y Yoo, AR Escobedo, R Kemmer, E Chiou - Scientific reports, 2024 - nature.com
We present a wisdom of crowds study where participants are asked to order a small set of
images based on the number of dots they contain and then to guess the respective number …

Enhancing image classification capabilities of crowdsourcing-based methods through expanded input elicitation

R Yasmin, JT Grassel, MM Hassan, O Fuentes… - Proceedings of the …, 2021 - ojs.aaai.org
This study investigates how different forms of input elicitation obtained from crowdsourcing
can be utilized to improve the quality of inferred labels for image classification tasks, where …

Assessing the effects of expanded input elicitation and machine learning-based priming on crowd stock prediction

H Bhogaraju, A Jain, J Jaiswal… - International Conference …, 2023 - Springer
The stock market is affected by a seemingly infinite number of factors, making it highly
uncertain yet impactful. A large determinant of stock performance is public sentiment, which …

Combining Human and AI Strengths in Object Counting under Information Asymmetry

S Liu, M Steyvers - Proceedings of the AAAI Conference on Human …, 2024 - ojs.aaai.org
With the recent development of artificial intelligence (AI), hybrid human-AI teams have
gained more attention and have been employed to solve all kinds of problems. However …