Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

V-SHiNE: A Virtual Smart Home Framework for Explainability Evaluation

Mersedeh Sadeghi, Simon Scholz, Max Unterbusch, Andreas Vogelsang

TL;DR

The paper tackles the challenge of evaluating explainability in smart-home CPS by highlighting the need for scalable, realistic, user-centered study platforms. It introduces V-SHiNE, a browser-based virtual smart home framework that lets researchers configure environments via JSON, plug in explanation engines, and collect rich interaction logs. The authors describe a modular architecture (frontend, backend, DB) with features such as TD/TDex-mapped devices, configurable automation rules, and multiple explanation delivery modes, supported by a Vitest-based testing regime to ensure reliability. Validation includes a 159-participant study and CIRCE-based scenario integration, illustrating the platform's feasibility and potential to accelerate rigorous, reproducible explainability research in CPS.

Abstract

Explanations are essential for helping users interpret and trust autonomous smart-home decisions, yet evaluating their quality and impact remains methodologically difficult in this domain. V-SHiNE addresses this gap: a browser-based smarthome simulation framework for scalable and realistic assessment of explanations. It allows researchers to configure environments, simulate behaviors, and plug in custom explanation engines, with flexible delivery modes and rich interaction logging. A study with 159 participants demonstrates its feasibility. V-SHiNE provides a lightweight, reproducible platform for advancing user-centered evaluation of explainable intelligent systems

V-SHiNE: A Virtual Smart Home Framework for Explainability Evaluation

TL;DR

The paper tackles the challenge of evaluating explainability in smart-home CPS by highlighting the need for scalable, realistic, user-centered study platforms. It introduces V-SHiNE, a browser-based virtual smart home framework that lets researchers configure environments via JSON, plug in explanation engines, and collect rich interaction logs. The authors describe a modular architecture (frontend, backend, DB) with features such as TD/TDex-mapped devices, configurable automation rules, and multiple explanation delivery modes, supported by a Vitest-based testing regime to ensure reliability. Validation includes a 159-participant study and CIRCE-based scenario integration, illustrating the platform's feasibility and potential to accelerate rigorous, reproducible explainability research in CPS.

Abstract

Explanations are essential for helping users interpret and trust autonomous smart-home decisions, yet evaluating their quality and impact remains methodologically difficult in this domain. V-SHiNE addresses this gap: a browser-based smarthome simulation framework for scalable and realistic assessment of explanations. It allows researchers to configure environments, simulate behaviors, and plug in custom explanation engines, with flexible delivery modes and rich interaction logging. A study with 159 participants demonstrates its feasibility. V-SHiNE provides a lightweight, reproducible platform for advancing user-centered evaluation of explainable intelligent systems
Paper Structure (7 sections, 2 figures)

This paper contains 7 sections, 2 figures.

Table of Contents

  1. Introduction & Motivation
  2. The V-SHiNE Framework
  3. Key Features
  4. Testing and Reliability Assurance
  5. Demo Walkthrough
  6. Validation & Early Experience
  7. Conclusion

Figures (2)

  • Figure 1: V-SHiNE interface layout.
  • Figure 2: High-level overview of the V-SHiNE framework.