Who validates the validators? aligning llm-assisted evaluation of llm outputs with human preferences

S Shankar, JD Zamfirescu-Pereira… - Proceedings of the 37th …, 2024 - dl.acm.org
Due to the cumbersome nature of human evaluation and limitations of code-based
evaluation, Large Language Models (LLMs) are increasingly being used to assist humans in …

Artificial intelligence co-piloted auditing

H Gu, M Schreyer, K Moffitt, M Vasarhelyi - International Journal of …, 2024 - Elsevier
This paper proposes the concept of artificial intelligence co-piloted auditing, emphasizing
the collaborative potential of auditors and foundation models in the auditing domain. The …

How do data analysts respond to ai assistance? a wizard-of-oz study

K Gu, M Grunde-McLaughlin, A McNutt, J Heer… - Proceedings of the CHI …, 2024 - dl.acm.org
Data analysis is challenging as analysts must navigate nuanced decisions that may yield
divergent conclusions. AI assistants have the potential to support analysts in planning their …

Improving steering and verification in AI-assisted data analysis with interactive task decomposition

M Kazemitabaar, J Williams, I Drosos… - Proceedings of the 37th …, 2024 - dl.acm.org
LLM-powered tools like ChatGPT Data Analysis, have the potential to help users tackle the
challenging task of data analysis programming, which requires expertise in data processing …

AI Should Challenge, Not Obey

A Sarkar - Communications of the ACM, 2024 - dl.acm.org
AI Should Challenge, Not Obey | Communications of the ACM skip to main content ACM Digital
Library home ACM Association for Computing Machinery corporate logo Google, Inc. (search) …

Beyond the chat: Executable and verifiable text-editing with llms

P Laban, J Vig, M Hearst, C **ong, CS Wu - Proceedings of the 37th …, 2024 - dl.acm.org
Conversational interfaces powered by Large Language Models (LLMs) have recently
become a popular way to obtain feedback during document editing. However, standard chat …

A framework for exploring the consequences of ai-mediated enterprise knowledge access and identifying risks to workers

A Gausen, B Mitra, S Lindley - The 2024 ACM Conference on Fairness …, 2024 - dl.acm.org
Organisations generate vast amounts of information, which has resulted in a long-term
research effort into knowledge access systems for enterprise settings. Recent developments …

Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models

MX Liu, T Wu, T Chen, FM Li, A Kittur… - Proceedings of the CHI …, 2024 - dl.acm.org
Sensemaking in unfamiliar domains can be challenging, demanding considerable user
effort to compare different options with respect to various criteria. Prior research and our …

Large Language Models Cannot Explain Themselves

A Sarkar - arxiv preprint arxiv:2405.04382, 2024 - arxiv.org
Large language models can be prompted to produce text. They can also be prompted to
produce" explanations" of their output. But these are not really explanations, because they …

Data Analysis in the Era of Generative AI

JP Inala, C Wang, S Drucker, G Ramos, V Dibia… - arxiv preprint arxiv …, 2024 - arxiv.org
This paper explores the potential of AI-powered tools to reshape data analysis, focusing on
design considerations and challenges. We explore how the emergence of large language …