Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GPTs Window Shopping: An analysis of the Landscape of Custom ChatGPT Models

Benjamin Zi Hao Zhao, Muhammad Ikram, Mohamed Ali Kaafar

TL;DR

The paper investigates OpenAI's GPTs Store by performing large-scale empirical analysis on Beetrove and EpicGPTstore datasets to map the landscape of user-created, prompt-tuned GPTs. It reveals strong English predominance, uneven use of official categories, and creation surges linked to public announcements, while highlighting extensive off-platform monetization and traffic via external hosting. The findings underscore ecosystem dynamics, including author diversity, engagement concentration, and potential risks related to prompt leakage and revenue leakage, informing platform governance and future store design. Overall, the work provides a baseline, data-backed view of how customizable LLM services are being produced, shared, and monetized in practice.

Abstract

OpenAI's ChatGPT initiated a wave of technical iterations in the space of Large Language Models (LLMs) by demonstrating the capability and disruptive power of LLMs. OpenAI has prompted large organizations to respond with their own advancements and models to push the LLM performance envelope. OpenAI has prompted large organizations to respond with their own advancements and models to push the LLM performance envelope. OpenAI's success in spotlighting AI can be partially attributed to decreased barriers to entry, enabling any individual with an internet-enabled device to interact with LLMs. What was previously relegated to a few researchers and developers with necessary computing resources is now available to all. A desire to customize LLMs to better accommodate individual needs prompted OpenAI's creation of the GPT Store, a central platform where users can create and share custom GPT models. Customization comes in the form of prompt-tuning, analysis of reference resources, browsing, and external API interactions, alongside a promise of revenue sharing for created custom GPTs. In this work, we peer into the window of the GPT Store and measure its impact. Our analysis constitutes a large-scale overview of the store exploring community perception, GPT details, and the GPT authors, in addition to a deep-dive into a 3rd party storefront indexing user-submitted GPTs, exploring if creators seek to monetize their creations in the absence of OpenAI's revenue sharing.

GPTs Window Shopping: An analysis of the Landscape of Custom ChatGPT Models

TL;DR

The paper investigates OpenAI's GPTs Store by performing large-scale empirical analysis on Beetrove and EpicGPTstore datasets to map the landscape of user-created, prompt-tuned GPTs. It reveals strong English predominance, uneven use of official categories, and creation surges linked to public announcements, while highlighting extensive off-platform monetization and traffic via external hosting. The findings underscore ecosystem dynamics, including author diversity, engagement concentration, and potential risks related to prompt leakage and revenue leakage, informing platform governance and future store design. Overall, the work provides a baseline, data-backed view of how customizable LLM services are being produced, shared, and monetized in practice.

Abstract

OpenAI's ChatGPT initiated a wave of technical iterations in the space of Large Language Models (LLMs) by demonstrating the capability and disruptive power of LLMs. OpenAI has prompted large organizations to respond with their own advancements and models to push the LLM performance envelope. OpenAI has prompted large organizations to respond with their own advancements and models to push the LLM performance envelope. OpenAI's success in spotlighting AI can be partially attributed to decreased barriers to entry, enabling any individual with an internet-enabled device to interact with LLMs. What was previously relegated to a few researchers and developers with necessary computing resources is now available to all. A desire to customize LLMs to better accommodate individual needs prompted OpenAI's creation of the GPT Store, a central platform where users can create and share custom GPT models. Customization comes in the form of prompt-tuning, analysis of reference resources, browsing, and external API interactions, alongside a promise of revenue sharing for created custom GPTs. In this work, we peer into the window of the GPT Store and measure its impact. Our analysis constitutes a large-scale overview of the store exploring community perception, GPT details, and the GPT authors, in addition to a deep-dive into a 3rd party storefront indexing user-submitted GPTs, exploring if creators seek to monetize their creations in the absence of OpenAI's revenue sharing.
Paper Structure (24 sections, 10 figures, 7 tables)

This paper contains 24 sections, 10 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Dataset
  3. Overall Analysis
  4. GPTs Community Perception
  5. Engagement
  6. Ratings
  7. Language
  8. GPTs Details
  9. Categories
  10. Capabilities
  11. Creation
  12. GPTs Authorship
  13. Author Contributions
  14. Author External Links
  15. 3rd Party Store Author Case Study
  16. ...and 9 more sections

Figures (10)

  • Figure 1: Cumulative distribution of community engagement with GPTs through the number of conversations with a GPT and the average rating of GPTs.
  • Figure 2: Distribution of GPTs creation dates. Notable events marked as 6 Nov 2023, and 10 Jan 2024 for OpenAI's Dev Day and the Official GPTs store launch respectively. Newest GPTs collected on 20 Jan 2024.
  • Figure 3: Cumulative distribution of GPTs submitted by the same author. Observe the long tail with a few authors submitting substantial numbers of GPTs.
  • Figure 4: Distribution of domain GPT authors and their creation date as recorded by whois information. November 2022 is the point in which ChatGPT was first launched by OpenAI.
  • Figure 5: Histogram of terms related to monetization, or "Blog" present within the webpages of our GPT domain authors.
  • ...and 5 more figures