Boosting Certified Robustness for Time Series Classification with Efficient Self-Ensemble

TL;DR

This work proposes a self-ensemble method to enhance the lower bound of the probability confidence of predicted labels by reducing the variance of classification margins, thereby certifying a larger radius.

Abstract

Recently, the issue of adversarial robustness in the time series domain has garnered significant attention. However, the available defense mechanisms remain limited, with adversarial training being the predominant approach, though it does not provide theoretical guarantees. Randomized Smoothing has emerged as a standout method due to its ability to certify a provable lower bound on robustness radius under $\ell_p$-ball attacks. Recognizing its success, research in the time series domain has started focusing on these aspects. However, existing research predominantly focuses on time series forecasting, or under the non-$\ell_p$ robustness in statistic feature augmentation for time series classification~(TSC). Our review found that Randomized Smoothing performs modestly in TSC, struggling to provide effective assurances on datasets with poor robustness. Therefore, we propose a self-ensemble method to enhance the lower bound of the probability confidence of predicted labels by reducing the variance of classification margins, thereby certifying a larger radius. This approach also addresses the computational overhead issue of Deep Ensemble~(DE) while remaining competitive and, in some cases, outperforming it in terms of robustness. Both theoretical analysis and experimental results validate the effectiveness of our method, demonstrating superior performance in robustness testing compared to baseline approaches.

Figures (6)

• Figure 1: Illustration of the Decision Boundary of Base Classifier $f$ over 3 Classes. In our method, the decision process is akin to randomly scattering points around $x$ and counting the number of points that fall into each region to determine the classification confidence. The first three diagrams illustrate the landscapes of three different base classifiers under different fixed masks. After ensembling, the combined base classifier performs better than a single classifier, with the area proportion of $p_A$ (where class $A$ is the top one output of the smoothed classifier) significantly increasing, thereby enhancing the certified radius. (For a detailed proof, please refer to the Theoretical Analysis in Section 4.)
• Figure 2: $L^2$ vs. $p_A$ for different values of $\alpha$. Certified radius can be prompted with increasing the $p_A$ in different $\alpha$.
• Figure 3: Effect of variance reduction on: Left) classification margins $z$. As the variance decreases through different ensembling methods, the probability $P(z > 0)$ increases, resulting in a higher certified radius. The red line represents the decision boundary at $z = 0$. Right) Corresponding prediction distribution of $c_A$. It exhibits a similar normal distribution, and a tightened distribution can be clearly observed in the ensemble. Additionally, the green line represents the mean. Except for DE, the expectation of the self-ensemble method is closer to the theoretical assumption compared to the Single. The subfigures show the distribution of these values for different ensembling methods: Single, DE, $M_B$, and $M_C$. (From ChlorineConcentration Dataset, $\sigma = 0.4$, 1000 noise samples)
• Figure 4: Certified Accuracy vs. radius in different $\sigma$ on ChlorineConcentration. ($\sigma$ is $0.2,\ 0.4,\ 0.8\ $and$\ 1.6$ from left to right)
• Figure 5: Certified accuracy vs. radius for different ensemble sizes on ChlorineConcentration.(Left: $M_B$, Right: $M_C$)