Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Who Are All The Stochastic Parrots Imitating? They Should Tell Us!

Sagi Shaier, Lawrence E. Hunter, Katharina von der Wense

TL;DR

It is argued that LMs in their current state will never be fully trustworthy in critical settings and a possible novel strategy to handle this issue is suggested: by building LMs such that can cite their sources - i.e., point a user to the parts of their training data that back up their outputs.

Abstract

Both standalone language models (LMs) as well as LMs within downstream-task systems have been shown to generate statements which are factually untrue. This problem is especially severe for low-resource languages, where training data is scarce and of worse quality than for high-resource languages. In this opinion piece, we argue that LMs in their current state will never be fully trustworthy in critical settings and suggest a possible novel strategy to handle this issue: by building LMs such that can cite their sources - i.e., point a user to the parts of their training data that back up their outputs. We first discuss which current NLP tasks would or would not benefit from such models. We then highlight the expected benefits such models would bring, e.g., quick verifiability of statements. We end by outlining the individual tasks that would need to be solved on the way to developing LMs with the ability to cite. We hope to start a discussion about the field's current approach to building LMs, especially for low-resource languages, and the role of the training data in explaining model generations.

Who Are All The Stochastic Parrots Imitating? They Should Tell Us!

TL;DR

It is argued that LMs in their current state will never be fully trustworthy in critical settings and a possible novel strategy to handle this issue is suggested: by building LMs such that can cite their sources - i.e., point a user to the parts of their training data that back up their outputs.

Abstract

Both standalone language models (LMs) as well as LMs within downstream-task systems have been shown to generate statements which are factually untrue. This problem is especially severe for low-resource languages, where training data is scarce and of worse quality than for high-resource languages. In this opinion piece, we argue that LMs in their current state will never be fully trustworthy in critical settings and suggest a possible novel strategy to handle this issue: by building LMs such that can cite their sources - i.e., point a user to the parts of their training data that back up their outputs. We first discuss which current NLP tasks would or would not benefit from such models. We then highlight the expected benefits such models would bring, e.g., quick verifiability of statements. We end by outlining the individual tasks that would need to be solved on the way to developing LMs with the ability to cite. We hope to start a discussion about the field's current approach to building LMs, especially for low-resource languages, and the role of the training data in explaining model generations.
Paper Structure (20 sections, 1 figure, 1 table)

This paper contains 20 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Factuality and the Lack Thereof
  4. Citation Generation
  5. Citations and Their Pros and Cons
  6. Which Tasks Require Citations?
  7. Benefits of Citations
  8. Risks of Citations
  9. Citations vs. Explainability
  10. Road Map
  11. The Big Picture
  12. Meta-information
  13. Retrieval
  14. When To Cite?
  15. Concrete Tasks to Master
  16. ...and 5 more sections

Figures (1)

  • Figure 1: An actual conversation with ChatGPT in Hebrew on the effects of not drinking enough water. ChatGPT is unable to point the user to its sources and instead falls back to a general answer (“I am ChatGPT, an OpenAI model based on the GPT-3.5 deep learning model. I am powered by OpenAI's learning set, which has been raised with the help of machine learning techniques on Internet culture, including websites, books, articles, quotes, and more”). We argue that ChatGPT and similar models should be able to direct the user to the sources of their information, which will have multiple benefits, such as quick verifiability of model statements.