Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Quantitative Study of SMS Phishing Detection

Daniel Timko, Daniel Hernandez Castillo, Muhammad Lutfor Rahman

TL;DR

This study investigates how users detect smishing in SMS by presenting 16 real or fake messages to 187 participants in a two-round online survey, augmented with a post-survey security-bias assessment. It demonstrates that users are more accurate at identifying fake messages (67.1%) than real ones (43.6%), and that attention to specific message regions (AOIs) and user characteristics predict detection performance. Key contributions include a systematic AOI framework for SMS content, demographic and behavioral analyses of interaction and comfort, and evidence-based recommendations for training and anti-smishing interfaces. The findings advance understanding of human factors in smishing defense and inform practical strategies to improve user resilience and messaging-app defenses against SMS phishing.

Abstract

With the booming popularity of smartphones, threats related to these devices are increasingly on the rise. Smishing, a combination of SMS (Short Message Service) and phishing has emerged as a treacherous cyber threat used by malicious actors to deceive users, aiming to steal sensitive information, money or install malware on their mobile devices. Despite the increase in smishing attacks in recent years, there are very few studies aimed at understanding the factors that contribute to a user's ability to differentiate real from fake messages. To address this gap in knowledge, we have conducted an online survey on smishing detection with 187 participants. In this study, we presented them with 16 SMS screenshots and evaluated how different factors affect their decision making process in smishing detection. Next, we conducted a post-survey to garner information on the participants' security attitudes, behavior and knowledge. Our results highlighted that attention and security behavioral scores had a significant impact on participants' accuracy in identifying smishing messages. We found that participants had more difficulty identifying real messages from fake ones, with an accuracy of 67.1% with fake messages and 43.6% with real messages. Our study is crucial in developing proactive strategies to encounter and mitigate smishing attacks. By understanding what factors influence smishing detection, we aim to bolster users' resilience against such threats and create a safer digital environment for all.

A Quantitative Study of SMS Phishing Detection

TL;DR

This study investigates how users detect smishing in SMS by presenting 16 real or fake messages to 187 participants in a two-round online survey, augmented with a post-survey security-bias assessment. It demonstrates that users are more accurate at identifying fake messages (67.1%) than real ones (43.6%), and that attention to specific message regions (AOIs) and user characteristics predict detection performance. Key contributions include a systematic AOI framework for SMS content, demographic and behavioral analyses of interaction and comfort, and evidence-based recommendations for training and anti-smishing interfaces. The findings advance understanding of human factors in smishing defense and inform practical strategies to improve user resilience and messaging-app defenses against SMS phishing.

Abstract

With the booming popularity of smartphones, threats related to these devices are increasingly on the rise. Smishing, a combination of SMS (Short Message Service) and phishing has emerged as a treacherous cyber threat used by malicious actors to deceive users, aiming to steal sensitive information, money or install malware on their mobile devices. Despite the increase in smishing attacks in recent years, there are very few studies aimed at understanding the factors that contribute to a user's ability to differentiate real from fake messages. To address this gap in knowledge, we have conducted an online survey on smishing detection with 187 participants. In this study, we presented them with 16 SMS screenshots and evaluated how different factors affect their decision making process in smishing detection. Next, we conducted a post-survey to garner information on the participants' security attitudes, behavior and knowledge. Our results highlighted that attention and security behavioral scores had a significant impact on participants' accuracy in identifying smishing messages. We found that participants had more difficulty identifying real messages from fake ones, with an accuracy of 67.1% with fake messages and 43.6% with real messages. Our study is crucial in developing proactive strategies to encounter and mitigate smishing attacks. By understanding what factors influence smishing detection, we aim to bolster users' resilience against such threats and create a safer digital environment for all.
Paper Structure (38 sections, 4 figures, 6 tables)

This paper contains 38 sections, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Phishing Detection User Studies
  4. Smishing Research
  5. Background
  6. Methodology
  7. Ethical Consideration And Mitigating Biases
  8. Recruitment and Demographics
  9. Real and Fake Messages Used
  10. Main Survey
  11. First Round Questions
  12. Second Round Questions
  13. Post-Survey
  14. Data Quality and Attention Checks
  15. Results
  16. ...and 23 more sections

Figures (4)

  • Figure 1: Breakdown of average metrics for handling each message, separated by real and fake messages.
  • Figure 2: Overview of our online survey protocol.
  • Figure 3: Accuracy for real and fake messages.
  • Figure 4: Messages used in experiment marked with areas of interest and click points. The first six screenshots are fake, the remaining nine are real.