Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards a Fully Interpretable and More Scalable RSA Model for Metaphor Understanding

Gaia Carenini, Luca Bischetti, Walter Schaeken, Valentina Bambini

TL;DR

This work targets interpretability and scalability gaps in Rational Speech Act models of metaphor understanding. It introduces a fully interpretable RSA framework with a closed-form distribution over communicative goals conditioned on topic and a gradient-based learning of the rationality parameter $\lambda$, enabling scalable training from limited data. Across 24 metaphors, the model exhibits strong alignment with human interpretations, especially for vehicle-inherent properties, and ablation confirms the importance of the context-sensitive goal term $\mathcal{R}(g|t)$ and parameter learning. The approach offers a principled bridge between classic pragmatic theories and modern optimization, with implications for broader pragmatic phenomena and potential insights into large language model metaphor comprehension.

Abstract

The Rational Speech Act (RSA) model provides a flexible framework to model pragmatic reasoning in computational terms. However, state-of-the-art RSA models are still fairly distant from modern machine learning techniques and present a number of limitations related to their interpretability and scalability. Here, we introduce a new RSA framework for metaphor understanding that addresses these limitations by providing an explicit formula - based on the mutually shared information between the speaker and the listener - for the estimation of the communicative goal and by learning the rationality parameter using gradient-based methods. The model was tested against 24 metaphors, not limited to the conventional $\textit{John-is-a-shark}$ type. Results suggest an overall strong positive correlation between the distributions generated by the model and the interpretations obtained from the human behavioral data, which increased when the intended meaning capitalized on properties that were inherent to the vehicle concept. Overall, findings suggest that metaphor processing is well captured by a typicality-based Bayesian model, even when more scalable and interpretable, opening up possible applications to other pragmatic phenomena and novel uses for increasing Large Language Models interpretability. Yet, results highlight that the more creative nuances of metaphorical meaning, not strictly encoded in the lexical concepts, are a challenging aspect for machines.

Towards a Fully Interpretable and More Scalable RSA Model for Metaphor Understanding

TL;DR

This work targets interpretability and scalability gaps in Rational Speech Act models of metaphor understanding. It introduces a fully interpretable RSA framework with a closed-form distribution over communicative goals conditioned on topic and a gradient-based learning of the rationality parameter , enabling scalable training from limited data. Across 24 metaphors, the model exhibits strong alignment with human interpretations, especially for vehicle-inherent properties, and ablation confirms the importance of the context-sensitive goal term and parameter learning. The approach offers a principled bridge between classic pragmatic theories and modern optimization, with implications for broader pragmatic phenomena and potential insights into large language model metaphor comprehension.

Abstract

The Rational Speech Act (RSA) model provides a flexible framework to model pragmatic reasoning in computational terms. However, state-of-the-art RSA models are still fairly distant from modern machine learning techniques and present a number of limitations related to their interpretability and scalability. Here, we introduce a new RSA framework for metaphor understanding that addresses these limitations by providing an explicit formula - based on the mutually shared information between the speaker and the listener - for the estimation of the communicative goal and by learning the rationality parameter using gradient-based methods. The model was tested against 24 metaphors, not limited to the conventional type. Results suggest an overall strong positive correlation between the distributions generated by the model and the interpretations obtained from the human behavioral data, which increased when the intended meaning capitalized on properties that were inherent to the vehicle concept. Overall, findings suggest that metaphor processing is well captured by a typicality-based Bayesian model, even when more scalable and interpretable, opening up possible applications to other pragmatic phenomena and novel uses for increasing Large Language Models interpretability. Yet, results highlight that the more creative nuances of metaphorical meaning, not strictly encoded in the lexical concepts, are a challenging aspect for machines.
Paper Structure (12 sections, 4 equations, 3 figures, 2 tables)

This paper contains 12 sections, 4 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Computational Model
  3. Behavioural Experiments
  4. Data availability statement
  5. Experiment 1. Feature Elicitation
  6. Experiment 2. Typicality Elicitation
  7. Experiment 3. Metaphor Understanding
  8. Model Evaluation
  9. Human vs. Model: Agreement
  10. Human vs Model: Correlation and (Dis)similarity
  11. Ablation study: Effect of introducing $\mathcal{R}(g|t)$ and learning
  12. Discussion

Figures (3)

  • Figure 1: Schematic representation of the RSA model for metaphor understanding. The model input is represented in the box on the top of the figure, and consists of topic-vehicle pairs (in our example, workers-ants) and associated values of typicality for a set of features (in our case, collaborative, strong, organized). Given this input, the model uses the parameter $\lambda$, which was previously learned from data (step 0 in the figure), in order to “stretch” the typicality values for the vehicle (e.g., to obtain $\beta_{\lambda}$) from $\beta$ (step 1). These values, which provide a proper estimator for the probability distribution over all the possible communicative goals, are combined with the typicality ones for the topic (e.g., $\alpha$; step 2) to build the final distribution over all the possible interpretations represented in the output box at the bottom of the figure.
  • Figure 2: Model vs. human interpretations: $k$-agreement, Pearson’s coefficient, Jensen-Shannon Divergence. In panel (a) and (b), the histograms report the frequency of overlaps of size $0,\dots, k$ for $k=1,3$ over the $k$ most common interpretations for the model and humans. The density curves represent the $k$-agreement distinguishing metaphors with vehicle-inherent and non vehicle-inherent properties. In panel (c) and (d), the histograms report the distributions of Pearson’s correlation coefficients (c) and Jensen-Shannon Divergence. The density curves represent the coefficients distinguishing metaphors with vehicle-inherent and non vehicle-inherent properties.
  • Figure 3: Correlations among features in metaphor interpretations in humans vs. RSA model. The table shows the correlations between pairs of features (translated from Italian into English) in the metaphor interpretations given by humans (below the diagonal) and by our model (above the diagonal). The colors of the cells represent the correlation values according to the scales on the right, (i.e., darker colors are associated with stronger correlation).