Z-Dip: a validated generalization of the Dip Test

TL;DR

The Z-Dip generalizes Hartigan’s Dip Test by standardizing the Dip statistic against its null distribution, yielding a scale-free multimodality score $Z\text{-}Dip = z = \frac{Dip_{obs}-\mu_N}{\sigma_N}$ with a universal threshold $z \approx 1.975$ calibrated via simulation and bootstrap. This removes sample-size dependence, preserves the original test’s nonparametric nature and $O(n \log n)$ complexity, and provides lookup tables for rapid evaluation. The approach is validated on synthetic Gaussian mixtures and $117{,}457$ real-world opinion distributions, showing close agreement with Dip-based decisions and robust performance across $N$, with a downsampling correction to mitigate large-sample sensitivity. The result is a practical, interpretable, and scalable tool for detecting and quantifying multimodality in diverse datasets, accompanied by open-source implementations. The work thereby facilitates consistent multimodality analysis across studies and applications where sample sizes vary substantially.

Abstract

Detecting multimodality in empirical distributions is a fundamental problem in statistics and data analysis, with applications ranging from clustering to social science. Hartigan's Dip Test is a classical nonparametric procedure for testing unimodality versus multimodality, but its interpretation is hindered by strong dependence on sample size and the need for lookup tables. We introduce the Z-Dip, a standardized extension of the Dip Test that removes sample-size dependence by comparing observed Dip values to simulated null distributions. We calibrate a universal decision threshold for Z-Dip via simulation and bootstrap resampling, providing a unified criterion for multimodality detection. In the final section, we also propose a downsampling-based approach to further mitigate residual sample-size effects in very large datasets. Lookup tables and software implementations are made available for efficient use in practice.

Figures (5)

• Figure 1: Comparison of the classical Dip statistic and the standardized Z-Dip across sample sizes. Z-Dip distributions were obtained over 9${,}$999 simulated uniform samples, with 1${,}$000 iterations of Z-Dip computation for each simulated sample. (A) Density of Z-Dip scores for different sample sizes ($N$), showing strong overlap of null distributions, proving that standardization removes the dependence on $N$. (B, C) Scatterplot of Dip statistic and Z-Dip values versus corresponding Dip Test $p$-values for simulated unimodal distributions, colored by sample size $N$. The strong monotonic relationship indicates that a single Z-Dip threshold consistently matches the $p < 0.05$ level across all $N$.
• Figure 2: Confusion matrix of the agreement between the Z-Dip threshold and the Dip Test's $p$-value over the $117{,}457$ empirical samples considered.
• Figure 3: Effect of sample size on Dip Test results and the corresponding downsampled Z-Dip correction. Both samples in the figure contain a detectable but practically negligible secondary mode around $X = 0.5$, which leads the test to classify the large sample as multimodal. The downsampled Z-Dip, however, correctly identifies the distribution as unimodal, illustrating its robustness to scale-induced artifacts.
• Figure 4: Stable trend of downsampled Z-Dip for sample sizes $N$ ranging from 150 to 72${,}$000, computed for data drawn from three distributions: (i) a bimodal Gaussian mixture with $\mu = (-0.6, 0.6)$, $\sigma = (0.1, 0.1)$, and equal component weights; (ii) a weakly bimodal distribution with $\mu = (-0.6, 0.6)$, $\sigma = (0.1, 0.2)$, and a minor mode weight of $0.025$; and (iii) a uniform distribution on $[0, 1]$. Each point represents the mean downsampled Z-Dip over $n_{sim} = 30$.
• Figure S1: Scaling of Z-Dip as $N$ grows. The average Z-Dip values were computed over 100 samples coming from the same underlying distribution, for each $N$.