Efficiently Sampling Interval Patterns from Numerical Databases

TL;DR

The paper tackles the challenge of exploring the enormous space of interval patterns in numerical databases by introducing two exact, non-enumerative sampling methods. Fips samples interval patterns in proportion to their frequency, while HFips extends this to the product of frequency and hyper-volume using IPH-based weighting. The authors provide formal proofs of the sampling distributions and demonstrate, across multiple datasets, that the methods produce higher-quality patterns, are more robust to long-tail effects, and offer favorable efficiency and diversity properties compared to uniform sampling. This work enables targeted, interactive exploration of interval-pattern spaces in numerical data and lays groundwork for future preference-aware sampling and broader applications in data mining and ML contexts.

Abstract

Pattern sampling has emerged as a promising approach for information discovery in large databases, allowing analysts to focus on a manageable subset of patterns. In this approach, patterns are randomly drawn based on an interestingness measure, such as frequency or hyper-volume. This paper presents the first sampling approach designed to handle interval patterns in numerical databases. This approach, named Fips, samples interval patterns proportionally to their frequency. It uses a multi-step sampling procedure and addresses a key challenge in numerical data: accurately determining the number of interval patterns that cover each object. We extend this work with HFips, which samples interval patterns proportionally to both their frequency and hyper-volume. These methods efficiently tackle the well-known long-tail phenomenon in pattern sampling. We formally prove that Fips and HFips sample interval patterns in proportion to their frequency and the product of hyper-volume and frequency, respectively. Through experiments on several databases, we demonstrate the quality of the obtained patterns and their robustness against the long-tail phenomenon.

Figures (14)

• Figure 1: Frequency evaluation for 500 patterns sampled with Fips and the Uniform methods
• Figure 2: Volume evaluation for 500 patterns sampled with HFips and the Uniform methods
• Figure 3: Diversity evaluation of Fips, HFips and Uniform methods
• Figure 4: Coverage diversity evaluation for 500 patterns sampled with Fips, HFips and the Uniform method
• Figure 5: CPU time evolution for a set of 500 patterns sampled by the Fips, HFips and Uniform methods

Theorems & Definitions (11)

• Example 1
• Example 2
• Example 3
• proof
• proof
• Example 4
• Example 5
• proof
• proof
• proof