The goldilocks of pragmatic understanding: Fine-tuning strategy matters for implicature resolution by llms

L Ruis, A Khan, S Biderman, S Hooker… - Advances in …, 2024 - proceedings.neurips.cc
Despite widespread use of LLMs as conversational agents, evaluations of performance fail
to capture a crucial aspect of communication: interpreting language in context …

Intentqa: Context-aware video intent reasoning

J Li, P Wei, W Han, L Fan - Proceedings of the IEEE/CVF …, 2023 - openaccess.thecvf.com
In this paper, we propose a novel task IntentQA, a special VideoQA task focusing on video
intent reasoning, which has become increasingly important for AI with its advantages in …

A fine-grained comparison of pragmatic language understanding in humans and language models

J Hu, S Floyd, O Jouravlev, E Fedorenko… - arxiv preprint arxiv …, 2022 - arxiv.org
Pragmatics and non-literal language understanding are essential to human communication,
and present a long-standing challenge for artificial language models. We perform a fine …

Large language models are not zero-shot communicators

LE Ruis, A Khan, S Biderman, S Hooker, T Rocktäschel… - 2022 - openreview.net
The recent success of large language models (LLMs) has drawn heavy attention and
investment in their use as conversational and embodied systems. Despite widespread use …

DiPlomat: a dialogue dataset for situated pragmatic reasoning

H Li, SC Zhu, Z Zheng - Advances in Neural Information …, 2023 - proceedings.neurips.cc
The ability to discern and comprehend pragmatic meanings is a cornerstone of social and
emotional intelligence, referred to as pragmatic reasoning. Despite the strides made in the …

Is the pope catholic? yes, the pope is catholic. generative evaluation of non-literal intent resolution in LLMs

A Yerukola, S Vaduguru, D Fried… - Proceedings of the 62nd …, 2024 - aclanthology.org
Humans often express their communicative intents indirectly or non-literally, which requires
their interlocutors—human or AI—to understand beyond the literal meaning of words. While …

Do large language models understand conversational implicature–a case study with a Chinese sitcom

S Yue, S Song, X Cheng, H Hu - China National Conference on Chinese …, 2024 - Springer
Understanding the non-literal meaning of an utterance is critical for large language models
(LLMs) to become human-like social communicators. In this work, we introduce …

Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering

N Srikanth, R Sarkar, H Mane, E Aparicio… - Proceedings of the …, 2024 - aclanthology.org
Questions posed by information-seeking users often contain implicit false or potentially
harmful assumptions. In a high-risk domain such as maternal and infant health, a question …

Entailed Between the Lines: Incorporating Implication into NLI

S Havaldar, H Alvari, J Palowitch, MJ Hosseini… - arxiv preprint arxiv …, 2025 - arxiv.org
Much of human communication depends on implication, conveying meaning beyond literal
words to express a wider range of thoughts, intentions, and feelings. For models to better …

Is the Pope Catholic? Yes, the Pope is Catholic. Generative Evaluation of Intent Resolution in LLMs

A Yerukola, S Vaduguru, D Fried, M Sap - arxiv preprint arxiv:2405.08760, 2024 - arxiv.org
Humans often express their communicative intents indirectly or non-literally, which requires
their interlocutors--human or AI--to understand beyond the literal meaning of words. While …