Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Statistical Significance and Discriminative Power in Pattern Discovery

Leonardo Alexandre, Rafael S. Costa, Rui Henriques

TL;DR

This work tackles pattern discovery in three-way tensor data by enforcing both statistical significance and discriminative power while preserving pattern quality. It introduces two components, Discriminative Power Component (DPC) and Statistical Significance Component (SSC), and couples them with an existing pattern quality criterion (PQC) via a Modified Objective Function (MOF) that can operate in additive or multiplicative form. The authors instantiate this framework on two triclustering algorithms, delta-Trimax and TriGen, and evaluate on three real-world datasets, showing consistent gains in discriminative power and statistical significance without sacrificing pattern quality. The approach is designed to be extendable to higher-order data structures and provides a pathway toward more actionable, supervised pattern discovery in complex data.

Abstract

Pattern discovery plays a central role in both descriptive and predictive tasks across multiple domains. Actionable patterns must meet rigorous statistical significance criteria and, in the presence of target variables, further uphold discriminative power. Our work addresses the underexplored area of guiding pattern discovery by integrating statistical significance and discriminative power criteria into state-of-the-art algorithms while preserving pattern quality. We also address how pattern quality thresholds, imposed by some algorithms, can be rectified to accommodate these additional criteria. To test the proposed methodology, we select the triclustering task as the guiding pattern discovery case and extend well-known greedy and multi-objective optimization triclustering algorithms, $δ$-Trimax and TriGen, that use various pattern quality criteria, such as Mean Squared Residual (MSR), Least Squared Lines (LSL), and Multi Slope Measure (MSL). Results from three case studies show the role of the proposed methodology in discovering patterns with pronounced improvements of discriminative power and statistical significance without quality deterioration, highlighting its importance in supervisedly guiding the search. Although the proposed methodology is motivated over multivariate time series data, it can be straightforwardly extended to pattern discovery tasks involving multivariate, N-way (N>3), transactional, and sequential data structures. Availability: The code is freely available at https://github.com/JupitersMight/MOF_Triclustering under the MIT license.

Integrating Statistical Significance and Discriminative Power in Pattern Discovery

TL;DR

This work tackles pattern discovery in three-way tensor data by enforcing both statistical significance and discriminative power while preserving pattern quality. It introduces two components, Discriminative Power Component (DPC) and Statistical Significance Component (SSC), and couples them with an existing pattern quality criterion (PQC) via a Modified Objective Function (MOF) that can operate in additive or multiplicative form. The authors instantiate this framework on two triclustering algorithms, delta-Trimax and TriGen, and evaluate on three real-world datasets, showing consistent gains in discriminative power and statistical significance without sacrificing pattern quality. The approach is designed to be extendable to higher-order data structures and provides a pathway toward more actionable, supervised pattern discovery in complex data.

Abstract

Pattern discovery plays a central role in both descriptive and predictive tasks across multiple domains. Actionable patterns must meet rigorous statistical significance criteria and, in the presence of target variables, further uphold discriminative power. Our work addresses the underexplored area of guiding pattern discovery by integrating statistical significance and discriminative power criteria into state-of-the-art algorithms while preserving pattern quality. We also address how pattern quality thresholds, imposed by some algorithms, can be rectified to accommodate these additional criteria. To test the proposed methodology, we select the triclustering task as the guiding pattern discovery case and extend well-known greedy and multi-objective optimization triclustering algorithms, -Trimax and TriGen, that use various pattern quality criteria, such as Mean Squared Residual (MSR), Least Squared Lines (LSL), and Multi Slope Measure (MSL). Results from three case studies show the role of the proposed methodology in discovering patterns with pronounced improvements of discriminative power and statistical significance without quality deterioration, highlighting its importance in supervisedly guiding the search. Although the proposed methodology is motivated over multivariate time series data, it can be straightforwardly extended to pattern discovery tasks involving multivariate, N-way (N>3), transactional, and sequential data structures. Availability: The code is freely available at https://github.com/JupitersMight/MOF_Triclustering under the MIT license.
Paper Structure (6 sections, 10 equations, 3 figures, 1 table)

This paper contains 6 sections, 10 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Methodology
  4. Computational Experimental Setup
  5. Results and Discussion
  6. Conclusion

Figures (3)

  • Figure 1: Example of standard lift and lift measurements in the context of four patterns and one class variable with three outcomes. The red pattern discriminates outcome A with a lift of $1.56$ and standardized lift of $0.5$, the blue pattern discriminates outcome C with a lift of $1.73$ and standardized lift of $0.5$, the green pattern discriminates outcome C with a lift of $2.16$ and standardized lift of $1$, the purple pattern discriminates outcome C with a lift of $2.16$ and standardized lift of $0.5$.
  • Figure 2: Illustration of the modified objective function based on additive approach, left column plots, multiplicative approach, right column plots. From top to bottom, each plot represents the distribution of values when considering: 1) all values, 2) values whose original p-values are less than $0.001$, 3) values whose DPC has a value less than $0.3$, and, 4) values with the two aforesaid restrictions. The x-axis presents the value returned by the modified objective function and the y-axis is the frequency of each value. To generate the values for the distribution a fixed PQC is used, $DPC \sim U(0,\ 1)$ and p-values are either sampled from $U(0,\ \theta)$ or $U(\theta,\ 1)$ with equal probability. Values to the left of the red line belong to the 5% lowest values of the distribution. Similarly, the yellow line marks the value of the 10th percentile.
  • Figure 3: Patterns and their corresponding discriminative power on outcomes of interest. For each pattern, each line represents how one of the variables in the pattern varies across time, with the x-axis representing the passage of time and the y-axis the measurement. The pattern example of the activity dataset was extracted by TriGen (MSR) with the additive approach. The pattern example of the Basketball dataset was extracted by TriGen (LSL) using the multiplicative approach. Finally, the pattern example for the sports dataset was extracted by $\delta$-Trimax using the additive approach. Variable description: Activity datasetmisc_activity_recognition_system_based_on_multisensor_data_fusion: -- Mean RSS (sensor 1-2) - mean received signal strength (RSS) between sensors 1 and 2; -- Mean RSS (sensor 2-3) - mean received signal strength (RSS) between sensors 2 and 3; -- Variance RSS (sensor 2-3) - variance received signal strength (RSS) between sensors 2 and 3. -- R(m/$s^2$) - gyroscope measurement. Basketball datasetmisc_basketball_dataset_587: -- Accelerometer (X-axis/Y-axis/Z-axis) - accelerometer measurement for specified axis; -- Gyroscope R measurement - gyroscope measurement. Sports datasetmisc_daily_and_sports_activities_256: -- Torso Accelerometer (X-axis) - torso accelerometer measurement for the x-axis; -- Torso Gyroscope (Y-axis), Torso Gyroscope (Z-axis) - torso gyroscopes measurements for the y and x-axis; -- Left Arm Gyroscope (Y-axis) - left arm gyroscope measurements for the y-axis.