CROCS: A Two-Stage Clustering Framework for Behaviour-Centric Consumer Segmentation with Smart Meter Data

TL;DR

CROCS tackles behavior-centric consumer segmentation with smart meter data by introducing a two-stage clustering framework that first builds per-consumer representations of daily load patterns (RLS) and then clusters consumers via the set-distance WSMD, which weights prototype prevalence. The framework also provides Refined RLS (RRLS) through graph-based community detection to summarize shared behavioral modes, enhancing interpretability. Empirical results on synthetic and real Australian datasets show CROCS captures intra-consumer variability, identifies asynchronous similarity (patterns that are similar but occur on different days), and remains robust to anomalies and missing data while scaling with parallelisation. The approach offers practical insights for DSM/DR design and can be extended to other domains requiring interpretable, scalable clustering of recurring time-series patterns.

Abstract

With grid operators confronting rising uncertainty from renewable integration and a broader push toward electrification, Demand-Side Management (DSM) -- particularly Demand Response (DR) -- has attracted significant attention as a cost-effective mechanism for balancing modern electricity systems. Unprecedented volumes of consumption data from a continuing global deployment of smart meters enable consumer segmentation based on real usage behaviours, promising to inform the design of more effective DSM and DR programs. However, existing clustering-based segmentation methods insufficiently reflect the behavioural diversity of consumers, often relying on rigid temporal alignment, and faltering in the presence of anomalies, missing data, or large-scale deployments. To address these challenges, we propose a novel two-stage clustering framework -- Clustered Representations Optimising Consumer Segmentation (CROCS). In the first stage, each consumer's daily load profiles are clustered independently to form a Representative Load Set (RLS), providing a compact summary of their typical diurnal consumption behaviours. In the second stage, consumers are clustered using the Weighted Sum of Minimum Distances (WSMD), a novel set-to-set measure that compares RLSs by accounting for both the prevalence and similarity of those behaviours. Finally, community detection on the WSMD-induced graph reveals higher-order prototypes that embody the shared diurnal behaviours defining consumer groups, enhancing the interpretability of the resulting clusters. Extensive experiments on both synthetic and real Australian smart meter datasets demonstrate that CROCS captures intra-consumer variability, uncovers both synchronous and asynchronous behavioural similarities, and remains robust to anomalies and missing data, while scaling efficiently through natural parallelisation. These results...

Figures (14)

• Figure 1: A visualisation of the WSMD set distance measure being applied to a pair of consumer representative load sets.
• Figure 2: A visualisation demonstrating computation of the RRLS hyperprototypes for consumer clusters.
• Figure 3: Mean performance and computational time (with pointwise 95 percentile intervals) of the CROCS framework over synthetic datasets with different numbers of true stage one clusters ($k^*$) and different numbers of outliers ($n_O$). (a) Recovery of consumer cluster labels according to ARI for HAC-Wa and KMd with different numbers of stage one clusters ($k$). (b) Computational time (in seconds) for both Stages of the CROCS framework.
• Figure 4: Mean reconstruction error (according to ED) for different load-profile based consumer representation methods across varying numbers of stage one clusters ($k$). Results are averaged across consumers and multiple 90-day periods: 24 periods for the AG dataset (left) and 10 periods for the SGSC dataset (right). Shaded regions show the full range of variation across the different time periods.
• Figure 5: Proportion of DLPs appointed to the largest cluster across different values of $k$ (displayed on a log-scale). The red line shows the mean proportion, with shaded regions representing the central 50%, 70%, and 90% intervals of the distribution. Results are based on all consumers from the AG and SGSC datasets over the same 90 day periods behind \ref{['Fig:Reconstruction_Error']}, but clustered using DTW-2 with either HAC-Wa or KMd algorithms to achieve more representative partitions. The grey dashed line indicates the expected proportion under uniform cluster size distribution for comparison.