A survey on evaluation of large language models

Y Chang, X Wang, J Wang, Y Wu, L Yang… - ACM Transactions on …, 2024 - dl.acm.org
Large language models (LLMs) are gaining increasing popularity in both academia and
industry, owing to their unprecedented performance in various applications. As LLMs …

Towards an understanding of large language models in software engineering tasks

Z Zheng, K Ning, Q Zhong, J Chen, W Chen… - Empirical Software …, 2025 - Springer
Abstract Large Language Models (LLMs) have drawn widespread attention and research
due to their astounding performance in text generation and reasoning tasks. Derivative …

The robots are here: Navigating the generative ai revolution in computing education

J Prather, P Denny, J Leinonen, BA Becker… - Proceedings of the …, 2023 - dl.acm.org
Recent advancements in artificial intelligence (AI) and specifically generative AI (GenAI) are
threatening to fundamentally reshape computing and society. Largely driven by large …

The promise and challenges of generative AI in education

M Giannakos, R Azevedo, P Brusilovsky… - Behaviour & …, 2024 - Taylor & Francis
Generative artificial intelligence (GenAI) tools, such as large language models (LLMs),
generate natural language and other types of content to perform a wide range of tasks. This …

The widening gap: The benefits and harms of generative ai for novice programmers

J Prather, BN Reeves, J Leinonen, S MacNeil… - Proceedings of the …, 2024 - dl.acm.org
Novice programmers often struggle through programming problem solving due to a lack of
metacognitive awareness and strategies. Previous research has shown that novices can …

A review of large language models and autonomous agents in chemistry

MC Ramos, CJ Collison, AD White - Chemical Science, 2025 - pubs.rsc.org
Large language models (LLMs) have emerged as powerful tools in chemistry, significantly
impacting molecule design, property prediction, and synthesis optimization. This review …

Codehelp: Using large language models with guardrails for scalable support in programming classes

M Liffiton, BE Sheese, J Savelka, P Denny - Proceedings of the 23rd Koli …, 2023 - dl.acm.org
Computing educators face significant challenges in providing timely support to students,
especially in large class settings. Large language models (LLMs) have emerged recently …

Enhancing llm-based feedback: Insights from intelligent tutoring systems and the learning sciences

J Stamper, R **ao, X Hou - International Conference on Artificial …, 2024 - Springer
Abstract The field of Artificial Intelligence in Education (AIED) focuses on the intersection of
technology, education, and psychology, placing a strong emphasis on supporting learners' …

Exploring the potential of large language models to generate formative programming feedback

N Kiesler, D Lohr, H Keuning - 2023 IEEE Frontiers in …, 2023 - ieeexplore.ieee.org
Ever since the emergence of large language models (LLMs) and related applications, such
as ChatGPT, its performance and error analysis for programming tasks have been subject to …

The Effects of Generative AI on Computing Students' Help-Seeking Preferences

I Hou, S Mettille, O Man, Z Li, C Zastudil… - Proceedings of the 26th …, 2024 - dl.acm.org
Help-seeking is a critical way that students learn new concepts, acquire new skills, and get
unstuck when problem-solving in their computing courses. The recent proliferation of …