Actionable Cybersecurity Notifications for Smart Homes: A User Study on the Role of Length and Complexity

TL;DR

This paper addresses the gap in translating technically dense IDS alerts into actionable, user-friendly notifications for smart homes. It adopts a controlled online study that systematically varies notification length (short vs long) and complexity (beginner, intermediate, expert) using six tailored prompt templates and GPT-4o to generate 60 unique notifications from Snort rules across 10 devices. The key finding is that intermediate-complexity notifications consistently outperform other combinations across likability, understandability, and motivation to act, with length effects differing by user expertise. The work provides concrete design guidelines and emphasizes a default to intermediate complexity with potential for adaptive messaging, contributing to more usable and effective security notifications in consumer smart-home environments.

Abstract

The proliferation of smart home devices has increased convenience but also introduced cybersecurity risks for everyday users, as many devices lack robust security features. Intrusion Detection Systems are a prominent approach to detecting cybersecurity threats. However, their alerts often use technical terms and require users to interpret them correctly, which is challenging for a typical smart home user. Large Language Models can bridge this gap by translating IDS alerts into actionable security notifications. However, it has not yet been clear what an actionable cybersecurity notification should look like. In this paper, we conduct an experimental online user study with 130 participants to examine how the length and complexity of LLM-generated notifications affect user likability, understandability, and motivation to act. Our results show that intermediate-complexity notifications are the most effective across all user groups, regardless of their technological proficiency. Across the board, users rated beginner-level messages as more effective when they were longer, while expert-level messages were rated marginally more effective when they were shorter. These findings provide insights for designing security notifications that are both actionable and broadly accessible to smart home users.

Figures (12)

• Figure 1: Short Intermediate Security Notification
• Figure 2: Wordcounts of all Security Notifications for each User-Length-Category
• Figure 3: Participants self-rated technological proficiency.
• Figure 4: Likability ratings by notification complexity and length, showing intermediate notifications as most likeable. Error bars represent standard errors of the mean.
• Figure 5: Understandability ratings by notification complexity and length, showing intermediate notifications as most understandable. Error bars represent standard errors of the mean.