Comparative Analysis of Hash-based Malware Clustering via K-Means

TL;DR

The paper evaluates three fuzzy-hash methods (SSDeep, TLSH, IMPHash) as features for K-means clustering of malware samples across labeled datasets. It constructs a custom PE malware dataset and uses fixed-length feature vectors to compare clustering quality, efficiency, and robustness. The study finds TLSH and IMPHash yield more semantically meaningful clusters, with IMPHash often performing best, while SSDeep is more efficient for broad categorization. Results inform threat-hunting pipelines on selecting and combining similarity hashing for scalable malware analysis.

Abstract

With the adoption of multiple digital devices in everyday life, the cyber-attack surface has increased. Adversaries are continuously exploring new avenues to exploit them and deploy malware. On the other hand, detection approaches typically employ hashing-based algorithms such as SSDeep, TLSH, and IMPHash to capture structural and behavioural similarities among binaries. This work focuses on the analysis and evaluation of these techniques for clustering malware samples using the K-means algorithm. More specifically, we experimented with established malware families and traits and found that TLSH and IMPHash produce more distinct, semantically meaningful clusters, whereas SSDeep is more efficient for broader classification tasks. The findings of this work can guide the development of more robust threat-detection mechanisms and adaptive security mechanisms.

Figures (4)

• Figure 1: Malware Family Distribution Samples
• Figure 2: Results of methods as clusters.
• Figure 3: Results of methods as families.
• Figure 4: Silhouette score of methods.