Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Commercial Anti-Smishing Tools and Their Comparative Effectiveness Against Modern Threats

Daniel Timko, Muhammad Lutfor Rahman

TL;DR

This study addresses the rising threat of SMS-based phishing by developing a test bed and releasing a public fresh-smishing dataset (smishtank) to benchmark commercial anti-smishing tools. It performs a three-way comparative evaluation across bulk messaging services, mobile carriers, and third-party anti-smishing apps using 20 smishing and 20 benign messages, and analyzes the blocking behavior at each layer. The results reveal substantial gaps: many tools provide limited additional protection beyond carrier filtering, with notable false positives in some apps and limited cross-tool overlap. The findings offer actionable insights for researchers and industry, highlighting the need for improved detection—especially zero-day smishing—across all three layers and suggesting concrete recommendations such as API access controls, explicit opt-in mechanisms, and ML-driven approaches.

Abstract

Smishing, also known as SMS phishing, is a type of fraudulent communication in which an attacker disguises SMS communications to deceive a target into providing their sensitive data. Smishing attacks use a variety of tactics; however, they have a similar goal of stealing money or personally identifying information (PII) from a victim. In response to these attacks, a wide variety of anti-smishing tools have been developed to block or filter these communications. Despite this, the number of phishing attacks continue to rise. In this paper, we developed a test bed for measuring the effectiveness of popular anti-smishing tools against fresh smishing attacks. To collect fresh smishing data, we introduce Smishtank.com, a collaborative online resource for reporting and collecting smishing data sets. The SMS messages were validated by a security expert and an in-depth qualitative analysis was performed on the collected messages to provide further insights. To compare tool effectiveness, we experimented with 20 smishing and benign messages across 3 key segments of the SMS messaging delivery ecosystem. Our results revealed significant room for improvement in all 3 areas against our smishing set. Most anti-phishing apps and bulk messaging services didn't filter smishing messages beyond the carrier blocking. The 2 apps that blocked the most smish also blocked 85-100\% of benign messages. Finally, while carriers did not block any benign messages, they were only able to reach a 25-35\% blocking rate for smishing messages. Our work provides insights into the performance of anti-smishing tools and the roles they play in the message blocking process. This paper would enable the research community and industry to be better informed on the current state of anti-smishing technology on the SMS platform.

Commercial Anti-Smishing Tools and Their Comparative Effectiveness Against Modern Threats

TL;DR

This study addresses the rising threat of SMS-based phishing by developing a test bed and releasing a public fresh-smishing dataset (smishtank) to benchmark commercial anti-smishing tools. It performs a three-way comparative evaluation across bulk messaging services, mobile carriers, and third-party anti-smishing apps using 20 smishing and 20 benign messages, and analyzes the blocking behavior at each layer. The results reveal substantial gaps: many tools provide limited additional protection beyond carrier filtering, with notable false positives in some apps and limited cross-tool overlap. The findings offer actionable insights for researchers and industry, highlighting the need for improved detection—especially zero-day smishing—across all three layers and suggesting concrete recommendations such as API access controls, explicit opt-in mechanisms, and ML-driven approaches.

Abstract

Smishing, also known as SMS phishing, is a type of fraudulent communication in which an attacker disguises SMS communications to deceive a target into providing their sensitive data. Smishing attacks use a variety of tactics; however, they have a similar goal of stealing money or personally identifying information (PII) from a victim. In response to these attacks, a wide variety of anti-smishing tools have been developed to block or filter these communications. Despite this, the number of phishing attacks continue to rise. In this paper, we developed a test bed for measuring the effectiveness of popular anti-smishing tools against fresh smishing attacks. To collect fresh smishing data, we introduce Smishtank.com, a collaborative online resource for reporting and collecting smishing data sets. The SMS messages were validated by a security expert and an in-depth qualitative analysis was performed on the collected messages to provide further insights. To compare tool effectiveness, we experimented with 20 smishing and benign messages across 3 key segments of the SMS messaging delivery ecosystem. Our results revealed significant room for improvement in all 3 areas against our smishing set. Most anti-phishing apps and bulk messaging services didn't filter smishing messages beyond the carrier blocking. The 2 apps that blocked the most smish also blocked 85-100\% of benign messages. Finally, while carriers did not block any benign messages, they were only able to reach a 25-35\% blocking rate for smishing messages. Our work provides insights into the performance of anti-smishing tools and the roles they play in the message blocking process. This paper would enable the research community and industry to be better informed on the current state of anti-smishing technology on the SMS platform.
Paper Structure (37 sections, 4 figures, 3 tables)

This paper contains 37 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Gathering Datasets
  4. Smishing Detection and Filtering
  5. Bulk messaging, ESME and Gateways
  6. Phishing Tool Analysis
  7. Background
  8. Methodology
  9. SMS Collection
  10. Data Characterization
  11. Squatting Techniques
  12. URL Types
  13. Named Entities
  14. Subcategories
  15. VirusTotal and Domain History
  16. ...and 22 more sections

Figures (4)

  • Figure 1: The mobile messaging ecosystem separated into different components.
  • Figure 2: The smish hit rate and benign strike rate of Messengers, Carriers and Apps. The right-hand upper corner of the figure indicates the best performance.
  • Figure 3: A visualization of messaging procedure.
  • Figure 4: The variables and their descriptions used in this research.