Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Search Engines Post-ChatGPT: How Generative Artificial Intelligence Could Make Search Less Reliable

Shahan Ali Memon, Jevin D. West

TL;DR

The evolving nature of search engines, as they begin to generate, index, and distribute content created by generative artificial intelligence (GenAI), is discussed, including how output from GenAI carries an unwarranted sense of credibility, while decreasing transparency and sourcing ability.

Abstract

In this commentary, we discuss the evolving nature of search engines, as they begin to generate, index, and distribute content created by generative artificial intelligence (GenAI). Our discussion highlights challenges in the early stages of GenAI integration, particularly around factual inconsistencies and biases. We discuss how output from GenAI carries an unwarranted sense of credibility, while decreasing transparency and sourcing ability. Furthermore, search engines are already answering queries with error-laden, generated content, further blurring the provenance of information and impacting the integrity of the information ecosystem. We argue how all these factors could reduce the reliability of search engines. Finally, we summarize some of the active research directions and open questions.

Search Engines Post-ChatGPT: How Generative Artificial Intelligence Could Make Search Less Reliable

TL;DR

The evolving nature of search engines, as they begin to generate, index, and distribute content created by generative artificial intelligence (GenAI), is discussed, including how output from GenAI carries an unwarranted sense of credibility, while decreasing transparency and sourcing ability.

Abstract

In this commentary, we discuss the evolving nature of search engines, as they begin to generate, index, and distribute content created by generative artificial intelligence (GenAI). Our discussion highlights challenges in the early stages of GenAI integration, particularly around factual inconsistencies and biases. We discuss how output from GenAI carries an unwarranted sense of credibility, while decreasing transparency and sourcing ability. Furthermore, search engines are already answering queries with error-laden, generated content, further blurring the provenance of information and impacting the integrity of the information ecosystem. We argue how all these factors could reduce the reliability of search engines. Finally, we summarize some of the active research directions and open questions.
Paper Structure (9 sections, 5 figures)

This paper contains 9 sections, 5 figures.

Table of Contents

  1. Generative search can make stuff up
  2. Generative search systems don't like to say 'No'
  3. Generative search engines obscure the provenance of information
  4. Generative search engines can reinforce biases
  5. The efficiency of generative search comes with the trade-off of its reliability
  6. Outlook
  7. Further reading
  8. About the authors
  9. Acknowledgment

Figures (5)

  • Figure 1: Searching controversial topics, such as abortion. (a) shows the search engine response to the query "problems with abortion" correctly citing the source. (b) shows the response to the query "problems with abo rt" mis-citing the source. Search results are from November 22, 2023.
  • Figure 2: Searching for benefits of nicotine. Search results are from December 4, 2023
  • Figure 3: Searching for "Jevin's theory of social echoes" on (a) Perplexity AI and (b) Arc Search. Almost none of this is true. Search results are from (a) January 18, 2024 and (b) February 15, 2024.
  • Figure 4: An example of how generative content online may affect search engine results. Courtesy: Delip Rao who shared this example on https://twitter.com/deliprao/status/1742235713293172942. Search results are from January 3, 2024.
  • Figure 5: Search results from Perplexity AI for the query "gift ideas for a 7 year old girl"(a) versus "gift ideas for a 7 year old boy"(b). Search results are from January 8, 2024.