Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns

Leon Hannig, Annika Bush, Meltem Aksoy, Steffen Becker, Greta Ontrup

TL;DR

The paper addresses how end-user-facing customizations of LLMaaS influence trust and usage patterns relative to commercial LLMs in university settings. It proposes a planned quantitative, cross-sectional field study at a large German university, comparing a customized LLMaaS (via OpenAI's ChatGPT on Azure with branding and UI changes) to ChatGPT, using a survey that measures trust, hallucinations, privacy, and sustainability, with a target of $N=250$ respondents. The study formulates hypotheses on trust elevation (H1–H2), hallucination-related behaviors (H3–H4), privacy perceptions (H5), and sustainable use (H6), aiming to provide actionable guidance for AI deployment in higher education. By examining organization-level cues and user perceptions, the work seeks to inform governance, UX design, and CSR-aligned AI strategy for institutional AI initiatives.

Abstract

As the use of Large Language Models (LLMs) by students, lecturers and researchers becomes more prevalent, universities - like other organizations - are pressed to develop coherent AI strategies. LLMs as-a-Service (LLMaaS) offer accessible pre-trained models, customizable to specific (business) needs. While most studies prioritize data, model, or infrastructure adaptations (e.g., model fine-tuning), we focus on user-salient customizations, like interface changes and corporate branding, which we argue influence users' trust and usage patterns. This study serves as a functional prequel to a large-scale field study in which we examine how students and employees at a German university perceive and use their institution's customized LLMaaS compared to ChatGPT. The goals of this prequel are to stimulate discussions on psychological effects of LLMaaS customizations and refine our research approach through feedback. Our forthcoming findings will deepen the understanding of trust dynamics in LLMs, providing practical guidance for organizations considering LLMaaS deployment.

Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns

TL;DR

The paper addresses how end-user-facing customizations of LLMaaS influence trust and usage patterns relative to commercial LLMs in university settings. It proposes a planned quantitative, cross-sectional field study at a large German university, comparing a customized LLMaaS (via OpenAI's ChatGPT on Azure with branding and UI changes) to ChatGPT, using a survey that measures trust, hallucinations, privacy, and sustainability, with a target of respondents. The study formulates hypotheses on trust elevation (H1–H2), hallucination-related behaviors (H3–H4), privacy perceptions (H5), and sustainable use (H6), aiming to provide actionable guidance for AI deployment in higher education. By examining organization-level cues and user perceptions, the work seeks to inform governance, UX design, and CSR-aligned AI strategy for institutional AI initiatives.

Abstract

As the use of Large Language Models (LLMs) by students, lecturers and researchers becomes more prevalent, universities - like other organizations - are pressed to develop coherent AI strategies. LLMs as-a-Service (LLMaaS) offer accessible pre-trained models, customizable to specific (business) needs. While most studies prioritize data, model, or infrastructure adaptations (e.g., model fine-tuning), we focus on user-salient customizations, like interface changes and corporate branding, which we argue influence users' trust and usage patterns. This study serves as a functional prequel to a large-scale field study in which we examine how students and employees at a German university perceive and use their institution's customized LLMaaS compared to ChatGPT. The goals of this prequel are to stimulate discussions on psychological effects of LLMaaS customizations and refine our research approach through feedback. Our forthcoming findings will deepen the understanding of trust dynamics in LLMs, providing practical guidance for organizations considering LLMaaS deployment.
Paper Structure (16 sections, 1 figure, 2 tables)

This paper contains 16 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. LLMs (as-a-Service) at Universities
  4. Trust in AI Systems
  5. LLM Hallucinations
  6. Data Security and Privacy Concerns
  7. Sustainable AI Use
  8. Method
  9. Study Design, Participants and Setting
  10. Customized LLMaaS
  11. Survey Instrument
  12. Trust
  13. Hallucinations
  14. Data security and privacy concerns
  15. Sustainable AI use
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Overview of our survey. The expected completion time is 15 minutes or less.