Implicature in Interaction: Understanding Implicature Improves Alignment in Human-LLM Interaction

TL;DR

The paper investigates whether understanding implicature can improve alignment in human–LLM interaction. It develops a theory-grounded taxonomy of three implicature classes and tests them across three experiments with six LLMs and human participants. Large models (notably GPT-4 family) approach human-like implicature interpretation, while prompting with implicature cues consistently boosts perceived relevance, quality, and user preference, especially for smaller models. The findings highlight practical, lightweight prompting benefits for HCI and call for architecture-level advances to achieve robust pragmatic competence and trustworthy AI interaction.

Abstract

The rapid advancement of Large Language Models (LLMs) is positioning language at the core of human-computer interaction (HCI). We argue that advancing HCI requires attention to the linguistic foundations of interaction, particularly implicature (meaning conveyed beyond explicit statements through shared context) which is essential for human-AI (HAI) alignment. This study examines LLMs' ability to infer user intent embedded in context-driven prompts and whether understanding implicature improves response generation. Results show that larger models approximate human interpretations more closely, while smaller models struggle with implicature inference. Furthermore, implicature-based prompts significantly enhance the perceived relevance and quality of responses across models, with notable gains in smaller models. Overall, 67.6% of participants preferred responses with implicature-embedded prompts to literal ones, highlighting a clear preference for contextually nuanced communication. Our work contributes to understanding how linguistic theory can be used to address the alignment problem by making HAI interaction more natural and contextually grounded.

Figures (12)

• Figure 1: Implicature in interaction. (a) Humans are adept at inferring implicatures during discussion. (b) Our experiment is designed to evaluate how large language models (LLMs) handle implicature-embedded prompts by asking humans to rate the responses based on their perceived relevance, quality, and preference., (c) In Experiment 1, larger models demonstrate a better understanding of conversational implicatures, approaching the human baseline., (d) With implicature embedded prompts, perceived relevance increases across all models however there are relevance differences between models.,(e) In experiment 3, we confirm that users consistently favor responses that are sensitive to implicatures over those that are not.
• Figure 2: Model accuracy relative to human baseline.
• Figure 3: Effects of model, intervention, and class on perceived relevance.
• Figure 4: Effects of model, intervention, and class on perceived quality.
• Figure 5: Caption for Experiment 3