Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Using ChatGPT to refine draft conceptual schemata in supply-driven design of multidimensional cubes

Stefano Rizzi

TL;DR

The paper investigates using GPT-4o to refine draft Dimensional Fact Model schemata produced by supply-driven design for multidimensional cubes. It employs YAML-based representations, structured prompts, and a five-stage experimental design to assess ChatGPT’s competencies in MD modeling, refinement, and the impact of prompt engineering. Baseline prompts yield multiple refinement errors, with common issues in syntax and hierarchy handling; carefully designed prompts plus iteration substantially reduce errors (from ~9 to ~4 on average), though some residual problems persist. The work demonstrates that LLMs can substantially support designers in conceptual refinement, enabling end-user participation, while still requiring human oversight for guaranteed validity and quality.

Abstract

Refinement is a critical step in supply-driven conceptual design of multidimensional cubes because it can hardly be automated. In fact, it includes steps such as the labeling of attributes as descriptive and the removal of uninteresting attributes, thus relying on the end-users' requirements on the one hand, and on the semantics of measures, dimensions, and attributes on the other. As a consequence, it is normally carried out manually by designers in close collaboration with end-users. The goal of this work is to check whether LLMs can act as facilitators for the refinement task, so as to let it be carried out entirely -- or mostly -- by end-users. The Dimensional Fact Model is the target formalism for our study; as a representative LLM, we use ChatGPT's model GPT-4o. To achieve our goal, we formulate three research questions aimed at (i) understanding the basic competences of ChatGPT in multidimensional modeling; (ii) understanding the basic competences of ChatGPT in refinement; and (iii) investigating if the latter can be improved via prompt engineering. The results of our experiments show that, indeed, a careful prompt engineering can significantly improve the accuracy of refinement, and that the residual errors can quickly be fixed via one additional prompt. However, we conclude that, at present, some involvement of designers in refinement is still necessary to ensure the validity of the refined schemata.

Using ChatGPT to refine draft conceptual schemata in supply-driven design of multidimensional cubes

TL;DR

The paper investigates using GPT-4o to refine draft Dimensional Fact Model schemata produced by supply-driven design for multidimensional cubes. It employs YAML-based representations, structured prompts, and a five-stage experimental design to assess ChatGPT’s competencies in MD modeling, refinement, and the impact of prompt engineering. Baseline prompts yield multiple refinement errors, with common issues in syntax and hierarchy handling; carefully designed prompts plus iteration substantially reduce errors (from ~9 to ~4 on average), though some residual problems persist. The work demonstrates that LLMs can substantially support designers in conceptual refinement, enabling end-user participation, while still requiring human oversight for guaranteed validity and quality.

Abstract

Refinement is a critical step in supply-driven conceptual design of multidimensional cubes because it can hardly be automated. In fact, it includes steps such as the labeling of attributes as descriptive and the removal of uninteresting attributes, thus relying on the end-users' requirements on the one hand, and on the semantics of measures, dimensions, and attributes on the other. As a consequence, it is normally carried out manually by designers in close collaboration with end-users. The goal of this work is to check whether LLMs can act as facilitators for the refinement task, so as to let it be carried out entirely -- or mostly -- by end-users. The Dimensional Fact Model is the target formalism for our study; as a representative LLM, we use ChatGPT's model GPT-4o. To achieve our goal, we formulate three research questions aimed at (i) understanding the basic competences of ChatGPT in multidimensional modeling; (ii) understanding the basic competences of ChatGPT in refinement; and (iii) investigating if the latter can be improved via prompt engineering. The results of our experiments show that, indeed, a careful prompt engineering can significantly improve the accuracy of refinement, and that the residual errors can quickly be fixed via one additional prompt. However, we conclude that, at present, some involvement of designers in refinement is still necessary to ensure the validity of the refined schemata.

Paper Structure

This paper contains 19 sections, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. LLMs for conceptual design
  4. Conceptual design of multidimensional cubes
  5. The investigation process
  6. Research questions
  7. Experiment design
  8. Base criteria and technology
  9. Input/output format
  10. Prompt templates
  11. Test cases
  12. Evaluation of the results
  13. Answer to RQ.1: Competences
  14. Discussion
  15. Answer to RQ.2: Refinement
  16. ...and 4 more sections

Figures (10)

  • Figure 1: YAML code to represent a draft DFM schema (left) and a refined one (right)
  • Figure 2: A DFM schema according to ChatGPT (left) and a real DFM schema (right)
  • Figure 3: Draft schema for test case C2 (only the first four letters of relation names are shown)
  • Figure 4: Partially refined schema for test case C2 with basic prompt (renaming and additivity)
  • Figure 5: Refined schema for test case C2 with basic prompt (all steps)
  • ...and 5 more figures