Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Am I Infected? Lessons from Operating a Large-Scale IoT Security Diagnostic Service

Takayuki Sasaki, Tomoya Inazawa, Youhei Yamaguchi, Simon Parkin, Michel van Eeten, Katsunari Yoshioka, Tsutomu Matsumoto

TL;DR

The paper evaluates am I infected?, a web-based IoT security diagnostic service launched in February 2022 for Japanese users, through 27 months of operation, a large user survey, and a remediation-focused study. It demonstrates that 114,747 users were diagnosed, with 585 users identified as having security issues (171 malware infections and 417 vulnerabilities) and remediation rates of 88% for malware and 50% for vulnerabilities among those re-diagnosed. Users praise the service for reassurance, low cost, and ease of use, while barriers to remediation include knowledge gaps, technical difficulty, and financial costs. The findings suggest that web-based diagnostics can effectively support remediation and complement traditional notification campaigns, but require attention to result detail, trust, and follow-up prompts to maximize impact.

Abstract

There is an expectation that users of home IoT devices will be able to secure those devices, but they may lack information about what they need to do. In February 2022, we launched a web service that scans users' IoT devices to determine how secure they are. The service aims to diagnose and remediate vulnerabilities and malware infections of IoT devices of Japanese users. This paper reports on findings from operating this service drawn from three studies: (1) the engagement of 114,747 users between February, 2022 - May, 2024; (2) a large-scale evaluation survey among service users (n=4,103), and; (3) an investigation and targeted survey (n=90) around the remediation actions of users of non-secure devices. During the operation, we notified 417 (0.36%) users that one or more of their devices were detected as vulnerable, and 171 (0.15%) users that one of their devices was infected with malware. The service found no issues for 99% of users. Still, 96% of all users evaluated the service positively, most often for it providing reassurance, being free of charge, and short diagnosis time. Of the 171 users with malware infections, 67 returned to the service later for a new check, with 59 showing improvement. Of the 417 users with vulnerable devices, 151 users revisited and re-diagnosed, where 75 showed improvement. We report on lessons learned, including a consideration of the capabilities that non-expert users will assume of a security scan.

Am I Infected? Lessons from Operating a Large-Scale IoT Security Diagnostic Service

TL;DR

The paper evaluates am I infected?, a web-based IoT security diagnostic service launched in February 2022 for Japanese users, through 27 months of operation, a large user survey, and a remediation-focused study. It demonstrates that 114,747 users were diagnosed, with 585 users identified as having security issues (171 malware infections and 417 vulnerabilities) and remediation rates of 88% for malware and 50% for vulnerabilities among those re-diagnosed. Users praise the service for reassurance, low cost, and ease of use, while barriers to remediation include knowledge gaps, technical difficulty, and financial costs. The findings suggest that web-based diagnostics can effectively support remediation and complement traditional notification campaigns, but require attention to result detail, trust, and follow-up prompts to maximize impact.

Abstract

There is an expectation that users of home IoT devices will be able to secure those devices, but they may lack information about what they need to do. In February 2022, we launched a web service that scans users' IoT devices to determine how secure they are. The service aims to diagnose and remediate vulnerabilities and malware infections of IoT devices of Japanese users. This paper reports on findings from operating this service drawn from three studies: (1) the engagement of 114,747 users between February, 2022 - May, 2024; (2) a large-scale evaluation survey among service users (n=4,103), and; (3) an investigation and targeted survey (n=90) around the remediation actions of users of non-secure devices. During the operation, we notified 417 (0.36%) users that one or more of their devices were detected as vulnerable, and 171 (0.15%) users that one of their devices was infected with malware. The service found no issues for 99% of users. Still, 96% of all users evaluated the service positively, most often for it providing reassurance, being free of charge, and short diagnosis time. Of the 171 users with malware infections, 67 returned to the service later for a new check, with 59 showing improvement. Of the 417 users with vulnerable devices, 151 users revisited and re-diagnosed, where 75 showed improvement. We report on lessons learned, including a consideration of the capabilities that non-expert users will assume of a security scan.
Paper Structure (32 sections, 8 figures, 12 tables)

This paper contains 32 sections, 8 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Web-based diagnosis service
  3. Design goals
  4. Scope of the service
  5. How to use the service
  6. Methodology
  7. Service implementation and deployment
  8. Participants
  9. Survey on user engagement
  10. User survey: obstacles to taking measures
  11. Ethical considerations
  12. Results: user survey
  13. Representative responses on good points
  14. Representative responses on bad points
  15. Responses of users rating the service as not helpful and users with security issues
  16. ...and 17 more sections

Figures (8)

  • Figure 1: Diagnosis result pages
  • Figure 2: Cumulative number of users and diagnoses.
  • Figure 3: Coding results of good and points about the service (red bars indicate the top 3 codes for good and bad points)
  • Figure 4: Time between notification and remediation confirmed
  • Figure 5: Questionnaire about engagement (Translated from Japanese)
  • ...and 3 more figures