Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancement of Subjective Content Descriptions by using Human Feedback

Magnus Bender, Tanya Braun, Ralf Möller, Marcel Gehrke

TL;DR

The paper tackles misalignment between user perceptions and SCD-based information retrieval by proposing ReFrESH, a relation-preserving, feedback-driven method to incrementally update SCDs without re-annotating the entire corpus. It combines a four-step update process (shift, disassemble, reassign, propagate) with FrESH for explicit feedback and Implicit Feedback Incorporation (IFI) to leverage ambiguous signals. The approach preserves inter-SCD relations while reconfiguring sentence-to-SCD assignments, enabling more user-specific SCD descriptions. An evaluation on German legal text shows that ReFrESH reduces divergence from a baseline and improves the consistency of SCD-based IR responses, demonstrating practical value for adaptive, user-aware IR systems.

Abstract

An agent providing an information retrieval service may work with a corpus of text documents. The documents in the corpus may contain annotations such as Subjective Content Descriptions (SCD) -- additional data associated with different sentences of the documents. Each SCD is associated with multiple sentences of the corpus and has relations among each other. The agent uses the SCDs to create its answers in response to queries supplied by users. However, the SCD the agent uses might reflect the subjective perspective of another user. Hence, answers may be considered faulty by an agent's user, because the SCDs may not exactly match the perceptions of an agent's user. A naive and very costly approach would be to ask each user to completely create all the SCD themselves. To use existing knowledge, this paper presents ReFrESH, an approach for Relation-preserving Feedback-reliant Enhancement of SCDs by Humans. An agent's user can give feedback about faulty answers to the agent. This feedback is then used by ReFrESH to update the SCDs incrementally. However, human feedback is not always unambiguous. Therefore, this paper additionally presents an approach to decide how to incorporate the feedback and when to update the SCDs. Altogether, SCDs can be updated with human feedback, allowing users to create even more specific SCDs for their needs.

Enhancement of Subjective Content Descriptions by using Human Feedback

TL;DR

The paper tackles misalignment between user perceptions and SCD-based information retrieval by proposing ReFrESH, a relation-preserving, feedback-driven method to incrementally update SCDs without re-annotating the entire corpus. It combines a four-step update process (shift, disassemble, reassign, propagate) with FrESH for explicit feedback and Implicit Feedback Incorporation (IFI) to leverage ambiguous signals. The approach preserves inter-SCD relations while reconfiguring sentence-to-SCD assignments, enabling more user-specific SCD descriptions. An evaluation on German legal text shows that ReFrESH reduces divergence from a baseline and improves the consistency of SCD-based IR responses, demonstrating practical value for adaptive, user-aware IR systems.

Abstract

An agent providing an information retrieval service may work with a corpus of text documents. The documents in the corpus may contain annotations such as Subjective Content Descriptions (SCD) -- additional data associated with different sentences of the documents. Each SCD is associated with multiple sentences of the corpus and has relations among each other. The agent uses the SCDs to create its answers in response to queries supplied by users. However, the SCD the agent uses might reflect the subjective perspective of another user. Hence, answers may be considered faulty by an agent's user, because the SCDs may not exactly match the perceptions of an agent's user. A naive and very costly approach would be to ask each user to completely create all the SCD themselves. To use existing knowledge, this paper presents ReFrESH, an approach for Relation-preserving Feedback-reliant Enhancement of SCDs by Humans. An agent's user can give feedback about faulty answers to the agent. This feedback is then used by ReFrESH to update the SCDs incrementally. However, human feedback is not always unambiguous. Therefore, this paper additionally presents an approach to decide how to incorporate the feedback and when to update the SCDs. Altogether, SCDs can be updated with human feedback, allowing users to create even more specific SCDs for their needs.
Paper Structure (28 sections, 2 equations, 9 figures, 7 algorithms)

This paper contains 28 sections, 2 equations, 9 figures, 7 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Notations
  5. Subjective Content Descriptions
  6. Supervised and UnSupervised Estimator for SCD Matrices
  7. Removing Sentences from Corpus and SCDs
  8. Relation-preserving Updates on SCD Matrices
  9. Relations to Preserve
  10. Four Steps for Updating an SCD
  11. Shift Relations to Sentences
  12. Disassemble SCD
  13. Reassign Sentences to SCDs
  14. Propagate new Relations
  15. Algorithm ReFrESH
  16. ...and 13 more sections

Figures (9)

  • Figure 1: Left: An SCD with its referenced sentences, three in this example, its row of the word distribution, and its additional data, e.g., containing a label and two relations to other SCDs. The red cross marks the sentence to remove from the SCD. Right: The SCD after the disassemble step. Each of the three sentences now stand by itself, while the label and the two relations have been reassigned to the individual sentences and given a factor.
  • Figure 2: Step 1) of ReFrESH -- Shift Relations to Sentences
  • Figure 3: Step 2) of ReFrESH -- Disassemble SCD
  • Figure 4: Step 3) of ReFrESH -- Reassign Sentences to SCDs
  • Figure 5: Step 4) of ReFrESH -- Propagate new Relations
  • ...and 4 more figures