Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cross-Cultural Validation of Partner Models for Voice User Interfaces

Katie Seaborn, Iona Gessinger, Suzuka Yoshida, Benjamin R. Cowan, Philip R. Doyle

TL;DR

This study addresses the lack of non-English measures for how users perceive voice user interfaces as dialogue partners by translating and validating the 18-item Partner Modelling Questionnaire (PMQ) into German (PMQ-DE) and Japanese (PMQ-JP). Using ITC-guided translation, pilot testing, and two large CFA-based online studies (German: n=185; Japanese: n=198), the authors demonstrate cross-cultural validity of the PMQ’s three-factor structure in both contexts, though communicative flexibility behaves differently in Japanese data. The work provides researchers with localized measurement tools for cross-cultural VUI research, discusses translation challenges (notably trust vs. reliance) and cultural nuances, and highlights implications for the design of conversational agents. By broadening linguistic and cultural coverage, this paper advances diversity and inclusion in HCI and VUI evaluation, and offers practical instruments for comparative UX studies across cultures.

Abstract

Recent research has begun to assess people's perceptions of voice user interfaces (VUIs) as dialogue partners, termed partner models. Current self-report measures are only available in English, limiting research to English-speaking users. To improve the diversity of user samples and contexts that inform partner modelling research, we translated, localized, and evaluated the Partner Modelling Questionnaire (PMQ) for non-English speaking Western (German, n=185) and East Asian (Japanese, n=198) cohorts where VUI use is popular. Through confirmatory factor analysis (CFA), we find that the scale produces equivalent levels of goodness-to-fit for both our German and Japanese translations, confirming its cross-cultural validity. Still, the structure of the communicative flexibility factor did not replicate directly across Western and East Asian cohorts. We discuss how our translations can open up critical research on cultural similarities and differences in partner model use and design, whilst highlighting the challenges for ensuring accurate translation across cultural contexts.

Cross-Cultural Validation of Partner Models for Voice User Interfaces

TL;DR

This study addresses the lack of non-English measures for how users perceive voice user interfaces as dialogue partners by translating and validating the 18-item Partner Modelling Questionnaire (PMQ) into German (PMQ-DE) and Japanese (PMQ-JP). Using ITC-guided translation, pilot testing, and two large CFA-based online studies (German: n=185; Japanese: n=198), the authors demonstrate cross-cultural validity of the PMQ’s three-factor structure in both contexts, though communicative flexibility behaves differently in Japanese data. The work provides researchers with localized measurement tools for cross-cultural VUI research, discusses translation challenges (notably trust vs. reliance) and cultural nuances, and highlights implications for the design of conversational agents. By broadening linguistic and cultural coverage, this paper advances diversity and inclusion in HCI and VUI evaluation, and offers practical instruments for comparative UX studies across cultures.

Abstract

Recent research has begun to assess people's perceptions of voice user interfaces (VUIs) as dialogue partners, termed partner models. Current self-report measures are only available in English, limiting research to English-speaking users. To improve the diversity of user samples and contexts that inform partner modelling research, we translated, localized, and evaluated the Partner Modelling Questionnaire (PMQ) for non-English speaking Western (German, n=185) and East Asian (Japanese, n=198) cohorts where VUI use is popular. Through confirmatory factor analysis (CFA), we find that the scale produces equivalent levels of goodness-to-fit for both our German and Japanese translations, confirming its cross-cultural validity. Still, the structure of the communicative flexibility factor did not replicate directly across Western and East Asian cohorts. We discuss how our translations can open up critical research on cultural similarities and differences in partner model use and design, whilst highlighting the challenges for ensuring accurate translation across cultural contexts.
Paper Structure (23 sections, 1 figure, 4 tables)

This paper contains 23 sections, 1 figure, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Localization and Cross-Cultural Research
  4. Concept and Measurement of Partner Models
  5. Research Aims
  6. Methods
  7. Stage 1: Translation
  8. Stage 2: Pilot Study
  9. Stage 3: Main Study
  10. Participants
  11. Procedure
  12. Data Analyses
  13. Assumption and Data Suitability Tests
  14. KMO, Bartlett's, and Determinate: German Sample
  15. KMO, Bartlett's, and Determinate: Japanese Sample
  16. ...and 8 more sections

Figures (1)

  • Figure 1: Path diagram of CFA outputs for the PMQ-EN, PMQ-DE, and PMQ-JP instruments, with factor loadings and between factor correlation coefficients.