Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data Analysis in the Era of Generative AI

Jeevana Priya Inala, Chenglong Wang, Steven Drucker, Gonzalo Ramos, Victor Dibia, Nathalie Riche, Dave Brown, Dan Marshall, Jianfeng Gao

TL;DR

This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges, and examines human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.

Abstract

This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow by translating high-level user intentions into executable code, charts, and insights. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps. Finally, we discuss the research challenges that impede the development of these AI-based systems such as enhancing model capabilities, evaluating and benchmarking, and understanding end-user needs.

Data Analysis in the Era of Generative AI

TL;DR

This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges, and examines human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps.

Abstract

This paper explores the potential of AI-powered tools to reshape data analysis, focusing on design considerations and challenges. We explore how the emergence of large language and multimodal models offers new opportunities to enhance various stages of data analysis workflow by translating high-level user intentions into executable code, charts, and insights. We then examine human-centered design principles that facilitate intuitive interactions, build user trust, and streamline the AI-assisted analysis workflow across multiple apps. Finally, we discuss the research challenges that impede the development of these AI-based systems such as enhancing model capabilities, evaluating and benchmarking, and understanding end-user needs.
Paper Structure (60 sections, 6 figures, 2 tables)

This paper contains 60 sections, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The Challenge of Data Analysis: A Complex and Iterative Process.
  3. The Generative AI (GenAI) revolution.
  4. Uniqueness of AI-powered tools for data analysis.
  5. Unlocking the potential of GenAI in data analysis.
  6. Background
  7. The data analysis process
  8. LLMs, LMMs, and agents
  9. Case study: User experiences with AI-tools for visualization authoring
  10. Example task and the traditional experience
  11. Conversational LLM interface (ChatGPT with GPT-4o and CodeInterpreter)
  12. LLM-powered interactive data analysis tools
  13. Remarks
  14. Opportunities for AI systems in the data analysis domain
  15. Closing the skills gap: Empowering users in data analysis
  16. ...and 45 more sections

Figures (6)

  • Figure 1: Overview of steps involved in deriving insights from data, highlighting the diverse skills required for these steps. We explicitly exclude any steps related to building/using models in this paper.
  • Figure 2: User experience for providing inputs and output formats with ChatGPT vs Data Formulator wang2023data
  • Figure 3: Constrasting conversational editing and iterations in (a) ChatGPT with (b) dynamic intent based UIs in DynaVis vaithilingam2024dynavis and (c) data threads in Data Formulator2 dataformulator2.
  • Figure 4: User experience for trust and verification of AI outputs in ChatGPT vs Data Formulator.
  • Figure 5: User experience for multi-app workflows with (a) existing AI assistants for individual apps vs (b) an OSAgent zhang2024ufo that transfers context between apps. Figure obtained from zhang2024ufo.
  • ...and 1 more figures