Crowdsourcing in computing education research: Case Amazon MTurk

A Hellas, A Zavgorodniaia, J Sorva - Proceedings of the 20th Koli Calling …, 2020 - dl.acm.org
Crowdsourcing platforms such as Amazon MTurk provide access to a human workforce that
can be given tasks to complete online for a fee. In this article, we review studies in …

Lessons learned from four computing education crowdsourcing systems

N Pirttinen, P Denny, A Hellas, J Leinonen - IEEE Access, 2023 - ieeexplore.ieee.org
Crowdsourcing is a general term that describes the practice of many individuals working
collectively to achieve a common goal or complete a task, often involving the generation of …

Can students review their peers? Comparison of peer and instructor reviews

N Pirttinen, J Leinonen - Proceedings of the 27th ACM Conference on on …, 2022 - dl.acm.org
Having students peer review each other's exercises is a common task in modern computing
classrooms. In large classes, peer review might even partly replace traditional instructor-led …

Crowdsourcing content creation for sql practice

J Leinonen, N Pirttinen, A Hellas - … of the 2020 ACM conference on …, 2020 - dl.acm.org
Crowdsourcing refers to the act of using the crowd to create content or to collect feedback on
some particular tasks or ideas. Within computer science education, crowdsourcing has been …

Bridging the gap between theory and empirical research in evaluative judgment.

H Khosrav, G Gyamf, BE Hanna, J Lodge… - Journal of Learning …, 2021 - ERIC
The value of students develo** the capacity to accurately judge the quality of their work
and that of others has been widely studied and recognized in higher education literature. To …

Fostering and supporting empirical research on evaluative judgement via a crowdsourced adaptive learning system

H Khosravi, G Gyamfi, BE Hanna, J Lodge - Proceedings of the Tenth …, 2020 - dl.acm.org
The value of students develo** the capacity to make accurate judgements about the
quality of their work and that of others has been widely recognised in higher education …

Could ChatGPT Be Used for Reviewing Learnersourced Exercises?

N Pirttinen, J Leinonen - Proceedings of the 23rd Koli Calling …, 2023 - dl.acm.org
Large language models and tools based on large language models such as ChatGPT have
received intense attention in the past year in computing education. In this work, we explore …

Does creating programming assignments with tests lead to improved performance in writing unit tests?

V Kangas, N Pirttinen, H Nygren, J Leinonen… - Proceedings of the …, 2019 - dl.acm.org
We have constructed a tool, CrowdSorcerer, in which students create programming
assignments, their model solutions and associated test cases using a simple input-output …

Experience report on a student-organized ai course

S Krings - Proceedings of the 27th ACM Conference on on …, 2022 - dl.acm.org
As curricula are mostly centered around the specializations of faculty members, students are
often seen purely as consumers of knowledge. In contrast, in the course presented in this …

[PDF][PDF] Exploring the complexity of crowdsourced programming assignments

N Pirttinen, J Leinonen - Seventh SPLICE Workshop at …, 2021 - researchportal.helsinki.fi
CrowdSorcerer is a tool in which students can create their own programming assignments
according to teacher's instructions, and later review their peers' assignments. In this lightning …