Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Faceted Evaluation of Modeling Languages for Augmented Reality Applications -- The Case of ARWFML

Fabian Muff, Hans-Georg Fill

TL;DR

The paper tackles the challenge of evaluating modeling languages for augmented reality in three-dimensional space. It advances ARWFML through design cycles that extend from a 2D implementation to a 3D notation and a dedicated 3D modeling environment (M2AR) to enable empirical assessments. It provides a comparative technical evaluation against related approaches, assesses ARWFML's coverage of standard workflow patterns, and reports an empirical comprehensibility study showing high understandability across participants. The findings underscore the importance of tool support and 3D-centric evaluation in maturing AR modeling languages for practical AR development.

Abstract

The evaluation of modeling languages for augmented reality applications poses particular challenges due to the three-dimensional environment they target. The previously introduced Augmented Reality Workflow Modeling Language (ARWFML) enables the model-based creation of augmented reality scenarios without programming knowledge. Building upon the first design cycle of the language's specification, this paper presents two further design iterations for refining the language based on multi-faceted evaluations. These include a comparative evaluation of implementation options and workflow capabilities, the introduction of a 3D notation, and the development of a new 3D modeling environment. On this basis, a comprehensibility study of the language was conducted. Thereby, we show how modeling languages for augmented reality can be evolved towards a maturity level suitable for empirical evaluations.

Multi-Faceted Evaluation of Modeling Languages for Augmented Reality Applications -- The Case of ARWFML

TL;DR

The paper tackles the challenge of evaluating modeling languages for augmented reality in three-dimensional space. It advances ARWFML through design cycles that extend from a 2D implementation to a 3D notation and a dedicated 3D modeling environment (M2AR) to enable empirical assessments. It provides a comparative technical evaluation against related approaches, assesses ARWFML's coverage of standard workflow patterns, and reports an empirical comprehensibility study showing high understandability across participants. The findings underscore the importance of tool support and 3D-centric evaluation in maturing AR modeling languages for practical AR development.

Abstract

The evaluation of modeling languages for augmented reality applications poses particular challenges due to the three-dimensional environment they target. The previously introduced Augmented Reality Workflow Modeling Language (ARWFML) enables the model-based creation of augmented reality scenarios without programming knowledge. Building upon the first design cycle of the language's specification, this paper presents two further design iterations for refining the language based on multi-faceted evaluations. These include a comparative evaluation of implementation options and workflow capabilities, the introduction of a 3D notation, and the development of a new 3D modeling environment. On this basis, a comprehensibility study of the language was conducted. Thereby, we show how modeling languages for augmented reality can be evolved towards a maturity level suitable for empirical evaluations.
Paper Structure (9 sections, 5 figures, 7 tables)

This paper contains 9 sections, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Outline of ARWFML and Comparative Evaluation of Related Approaches in the Second Design Cycle
  3. Comparative Evaluation of Technical Implementation
  4. Evaluation of Workflow Capabilities
  5. Third Design Cycle: Towards Empirical Evaluations
  6. Preliminaries: Adaptation of ARWFML to 3D
  7. The 3D Modeling Environment M2AR
  8. Empirical User Study for ARWFML Model Comprehensibility
  9. Conclusion and Outlook

Figures (5)

  • Figure 1: Design cycles for advancing the ARWFML language muff2023domain, following Design Science Research Methodsmorana2019designing. The light gray areas have already been addressed.
  • Figure 2: 2D notation of of the ARWFML language concepts, separated by the three different language types ObjectSpace, Statechange, and FlowScene.
  • Figure 3: ARWFML FlowScene model illustrating the eight supported workflow patterns by ARWFML. Each pattern in the model is marked with a red circle indicating the number of the workflow pattern according to Table \ref{['tab:workflow_patterns']}. (1) Sequence; (2) Parallel Split; (5) Simple Merge; (6) Multi-choice; (8) Multi-merge; (10) Arbitrary Cycles; (11) Implicit Termination; (20) Cancel Case.
  • Figure 4: 3D notation of of the ARWFML language concepts grouped by the SceneTypesObjectSpace, Statechange, and FlowScene. For each SceneType, the visual notation is shown in the 3D Notation column. If there is a dynamic visualization depending on attribute values, an example visualization appears in the Dynamic Example column.
  • Figure 5: Visual comparison of ObjectSpace, Statechange, and FlowScene models in (a.) 2D notation of ARWFML in the ADOxx modeling tool from the first design cycle and (b.) the new 3D notation in the new M2AR modeling client.