Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring the Innovation Opportunities for Pre-trained Models

Minjung Park, Jodi Forlizzi, John Zimmerman

TL;DR

This paper addresses the challenge that innovators lack a clear sense of where pre-trained models (LLMs, Generative AI, Foundation Models) reliably create value. It adopts a designerly approach, treating pre-trained models as materials and analyzing 85 ACM CHI/DIS artifacts to map domains, capabilities, required task-expertise, and performance, as well as emerging interaction patterns. The study presents a first draft resource detailing domains with value, a 294-capability taxonomy (organized into 33 clusters, 13 actions, and 3 themes), and seven interaction design patterns, offering guidance for starting points in AI product ideation and design while highlighting gaps for future work. These insights support innovators in choosing lower-risk applications, though the analysis acknowledges limitations such as the focus on research papers and the absence of a financial-viability assessment, which invites further validation across commercial contexts.

Abstract

Innovators transform the world by understanding where services are successfully meeting customers' needs and then using this knowledge to identify failsafe opportunities for innovation. Pre-trained models have changed the AI innovation landscape, making it faster and easier to create new AI products and services. Understanding where pre-trained models are successful is critical for supporting AI innovation. Unfortunately, the hype cycle surrounding pre-trained models makes it hard to know where AI can really be successful. To address this, we investigated pre-trained model applications developed by HCI researchers as a proxy for commercially successful applications. The research applications demonstrate technical capabilities, address real user needs, and avoid ethical challenges. Using an artifact analysis approach, we categorized capabilities, opportunity domains, data types, and emerging interaction design patterns, uncovering some of the opportunity space for innovation with pre-trained models.

Exploring the Innovation Opportunities for Pre-trained Models

TL;DR

This paper addresses the challenge that innovators lack a clear sense of where pre-trained models (LLMs, Generative AI, Foundation Models) reliably create value. It adopts a designerly approach, treating pre-trained models as materials and analyzing 85 ACM CHI/DIS artifacts to map domains, capabilities, required task-expertise, and performance, as well as emerging interaction patterns. The study presents a first draft resource detailing domains with value, a 294-capability taxonomy (organized into 33 clusters, 13 actions, and 3 themes), and seven interaction design patterns, offering guidance for starting points in AI product ideation and design while highlighting gaps for future work. These insights support innovators in choosing lower-risk applications, though the analysis acknowledges limitations such as the focus on research papers and the absence of a financial-viability assessment, which invites further validation across commercial contexts.

Abstract

Innovators transform the world by understanding where services are successfully meeting customers' needs and then using this knowledge to identify failsafe opportunities for innovation. Pre-trained models have changed the AI innovation landscape, making it faster and easier to create new AI products and services. Understanding where pre-trained models are successful is critical for supporting AI innovation. Unfortunately, the hype cycle surrounding pre-trained models makes it hard to know where AI can really be successful. To address this, we investigated pre-trained model applications developed by HCI researchers as a proxy for commercially successful applications. The research applications demonstrate technical capabilities, address real user needs, and avoid ethical challenges. Using an artifact analysis approach, we categorized capabilities, opportunity domains, data types, and emerging interaction design patterns, uncovering some of the opportunity space for innovation with pre-trained models.

Paper Structure

This paper contains 33 sections, 12 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. HCI Investigations of Practitioners
  4. Resources and Methods for Improving Ideation
  5. Innovation with Pre-trained Models
  6. Method
  7. Selecting a Corpus
  8. Artifact Analysis
  9. Identify Domains
  10. Extract Capabilities
  11. Infer Expertise and Performance
  12. Explore Interaction Design Patterns
  13. Findings
  14. Domains Where Pre-trained Models Create Value
  15. Pre-trained Model Capabilities
  16. ...and 18 more sections

Figures (12)

  • Figure 1: The number of papers presenting applications made with pre-trained models over the past three years from CHI, DIS, UIST and CSCW. We start in 2022 as this is when Open AI released ChatGPT and kicked off the public's interest in pre-trained models.
  • Figure 2: Illustrating the search, filtering, inclusion, and exclusion process.
  • Figure 3: Mapping of pretrained model applications to industrial domains published in Yildirim et alyildirim2023creating where AI has traditionally created value.
  • Figure 4: Task-expertise and Model-performance
  • Figure 5: Examples of Chatbot Interview
  • ...and 7 more figures