Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving RAG Retrieval via Propositional Content Extraction: a Speech Act Theory Approach

João Alberto de Oliveira Lima

TL;DR

The paper tackles embedding mismatches in Retrieval-Augmented Generation by separating the propositional content from illocutionary markers in user queries. It introduces a GPT-4 guided propositional-content extraction framework that reformulates queries into content-focused declaratives before embedding. Across a Brazilian telecom news corpus, reformulated queries yield higher semantic similarity between query and document embeddings at top ranks, with notable gains for commissive, expressive, indirect, interrogative, and directive speech acts. This approach provides a lightweight, non-generative preprocessing step that improves retrieval alignment and has potential for generalization across domains and languages, though it currently relies on domain-specific data and qualitative evaluation metrics.

Abstract

When users formulate queries, they often include not only the information they seek, but also pragmatic markers such as interrogative phrasing or polite requests. Although these speech act indicators communicate the user\textquotesingle s intent -- whether it is asking a question, making a request, or stating a fact -- they do not necessarily add to the core informational content of the query itself. This paper investigates whether extracting the underlying propositional content from user utterances -- essentially stripping away the linguistic markers of intent -- can improve retrieval quality in Retrieval-Augmented Generation (RAG) systems. Drawing upon foundational insights from speech act theory, we propose a practical method for automatically transforming queries into their propositional equivalents before embedding. To assess the efficacy of this approach, we conducted an experimental study involving 63 user queries related to a Brazilian telecommunications news corpus with precomputed semantic embeddings. Results demonstrate clear improvements in semantic similarity between query embeddings and document embeddings at top ranks, confirming that queries stripped of speech act indicators more effectively retrieve relevant content.

Improving RAG Retrieval via Propositional Content Extraction: a Speech Act Theory Approach

TL;DR

The paper tackles embedding mismatches in Retrieval-Augmented Generation by separating the propositional content from illocutionary markers in user queries. It introduces a GPT-4 guided propositional-content extraction framework that reformulates queries into content-focused declaratives before embedding. Across a Brazilian telecom news corpus, reformulated queries yield higher semantic similarity between query and document embeddings at top ranks, with notable gains for commissive, expressive, indirect, interrogative, and directive speech acts. This approach provides a lightweight, non-generative preprocessing step that improves retrieval alignment and has potential for generalization across domains and languages, though it currently relies on domain-specific data and qualitative evaluation metrics.

Abstract

When users formulate queries, they often include not only the information they seek, but also pragmatic markers such as interrogative phrasing or polite requests. Although these speech act indicators communicate the user\textquotesingle s intent -- whether it is asking a question, making a request, or stating a fact -- they do not necessarily add to the core informational content of the query itself. This paper investigates whether extracting the underlying propositional content from user utterances -- essentially stripping away the linguistic markers of intent -- can improve retrieval quality in Retrieval-Augmented Generation (RAG) systems. Drawing upon foundational insights from speech act theory, we propose a practical method for automatically transforming queries into their propositional equivalents before embedding. To assess the efficacy of this approach, we conducted an experimental study involving 63 user queries related to a Brazilian telecommunications news corpus with precomputed semantic embeddings. Results demonstrate clear improvements in semantic similarity between query embeddings and document embeddings at top ranks, confirming that queries stripped of speech act indicators more effectively retrieve relevant content.

Paper Structure

This paper contains 12 sections, 4 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Speech Act Theory and Computational Applications
  4. Retrieval-Augmented Generation (RAG) and Vector Embeddings
  5. Methodology
  6. Propositional Content Extraction from Queries
  7. Experimental Setup
  8. Example of the Process
  9. Results and Discussion
  10. Conclusion
  11. Appendix
  12. Presentation and Interpretation of Query Metrics

Figures (4)

  • Figure 1: Comparison between the standard retrieval approach and our propositional content extraction method.
  • Figure 2: Average Character Reduction per Speech Act Type after Propositional Content Extraction
  • Figure 3: Comparison of Mean Similarity Between Original and Propositional Query Forms by Speech Act Type and Query Number. Colors indicate change: green = no change; blue = improved similarity; red = decreased similarity.
  • Figure 4: Comparison of Similarity Distributions Between Original and Propositional Query Forms by Speech Act Type