Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions

F Daniel, P Kucherbaev, C Cappiello… - ACM Computing …, 2018 - dl.acm.org
Crowdsourcing enables one to leverage on the intelligence and wisdom of potentially large
groups of individuals toward solving problems. Common problems approached with …

A survey on task assignment in crowdsourcing

D Hettiachchi, V Kostakos, J Goncalves - ACM Computing Surveys …, 2022 - dl.acm.org
Quality improvement methods are essential to gathering high-quality crowdsourced data,
both for research and industry applications. A popular and broadly applicable method is task …

On creating benchmark dataset for aerial image interpretation: Reviews, guidances, and million-aid

Y Long, GS **a, S Li, W Yang, MY Yang… - IEEE Journal of …, 2021 - ieeexplore.ieee.org
The past years have witnessed great progress on remote sensing (RS) image interpretation
and its wide applications. With RS images becoming more accessible than ever before …

Comparing data characteristics and results of an online factorial survey between a population-based and a crowdsource-recruited sample

J Weinberg, J Freese… - Sociological Science, 2014 - sociologicalscience.com
Compared to older kinds of sample surveys, online platforms provide a fast and low-cost
platform for factorial surveys, as well as a more demographically diverse alternative to …

[HTML][HTML] The quality of data collected online: An investigation of careless responding in a crowdsourced sample

F Brühlmann, S Petralito, LF Aeschbach, K Opwis - Methods in Psychology, 2020 - Elsevier
Despite recent concerns about data quality, various academic fields rely increasingly on
crowdsourced samples. Thus, the goal of this study was to systematically assess …

Understanding malicious behavior in crowdsourcing platforms: The case of online surveys

U Gadiraju, R Kawase, S Dietze… - Proceedings of the 33rd …, 2015 - dl.acm.org
Crowdsourcing is increasingly being used as a means to tackle problems requiring human
intelligence. With the ever-growing worker base that aims to complete microtasks on …

Cognitive biases in crowdsourcing

C Eickhoff - Proceedings of the eleventh ACM international …, 2018 - dl.acm.org
Crowdsourcing has become a popular paradigm in data curation, annotation and evaluation
for many artificial intelligence and information retrieval applications. Considerable efforts …

An evaluation of aggregation techniques in crowdsourcing

N Quoc Viet Hung, NT Tam, LN Tran… - … Conference on Web …, 2013 - Springer
As the volumes of AI problems involving, human knowledge are likely to soar,
crowdsourcing has become essential in a wide range of world-wide-web applications. One …

Rehumanized crowdsourcing: A labeling framework addressing bias and ethics in machine learning

NM Barbosa, M Chen - Proceedings of the 2019 CHI Conference on …, 2019 - dl.acm.org
The increased use of machine learning in recent years led to large volumes of data being
manually labeled via crowdsourcing microtasks completed by humans. This brought about …

A survey of general-purpose crowdsourcing techniques

AI Chittilappilly, L Chen… - IEEE Transactions on …, 2016 - ieeexplore.ieee.org
Since Jeff Howe introduced the term Crowdsourcing in 2006, this human-powered problem-
solving paradigm has gained a lot of attention and has been a hot research topic in the field …