Advancing Out-of-Distribution Detection via Local Neuroplasticity

TL;DR

This work tackles the challenge of detecting out-of-distribution data by exploiting the local neuroplasticity of Kolmogorov-Arnold Networks (KANs). The authors propose a post-hoc detector that compares activation patterns between a trained KAN and an identical untrained copy, using the differences to distinguish InD from OOD samples, and extend this with a joint-distribution capture mechanism via dataset partitioning. Empirical results across seven benchmarks in image and medical domains show state-of-the-art AUROC and robustness to training-set size, demonstrating the practical viability of KAN-based OOD detection. The approach emphasizes interpretability and leverages spline-based activations to achieve strong performance while remaining adaptable to diverse backbones and data regimes.

Abstract

In the domain of machine learning, the assumption that training and test data share the same distribution is often violated in real-world scenarios, requiring effective out-of-distribution (OOD) detection. This paper presents a novel OOD detection method that leverages the unique local neuroplasticity property of Kolmogorov-Arnold Networks (KANs). Unlike traditional multilayer perceptrons, KANs exhibit local plasticity, allowing them to preserve learned information while adapting to new tasks. Our method compares the activation patterns of a trained KAN against its untrained counterpart to detect OOD samples. We validate our approach on benchmarks from image and medical domains, demonstrating superior performance and robustness compared to state-of-the-art techniques. These results underscore the potential of KANs in enhancing the reliability of machine learning systems in diverse environments.

Figures (5)

• Figure 1: Overview of the proposed method: the detector compares the activation function response of a trained KAN model with its untrained counterpart. A difference in the response indicates the sample is InD, a similar response suggests it is OOD.
• Figure 2: (a) Visualization of the training dataset, showing the relationship between inputs and targets. (b) Response of the untrained KAN model across the entire input range. (c) Response of the trained KAN model across the entire input range. (d) Test dataset illustrating inputs versus targets, created by combining the training dataset (InD) with three additional Gaussian peaks over the remaining input range (OOD). (e) InD score $S(\textbf{x}) \forall \textbf{x} \in [-1, 1]$ using the median as scoring function ($F_{\text{score}}$). (f) Final results after applying a threshold ($\lambda=1e-3$) to the InD scores: blue regions indicate predicted InD areas and red regions indicate predicted OOD areas.
• Figure 3: 2D L-shape toy dataset: The blue point cloud shows the training distribution and the red points are OOD test samples. The black contour represents the thresholded score function (median) separating InD from OOD samples. (a) Default KAN detector limited by the marginal distribution of $x_1$ and $x_2$. (b) Improved performance by partitioning the training dataset with KMeans clustering ($\mathcal{P}=2$) and $F_{\text{agg.}}=\text{max}$ as aggregation function.
• Figure 4: Distribution of activation's differences ($\Delta$) for three different samples (InD, near and far OOD). The InD sample tends to produce bigger values in the $\Delta$ matrix compared to the OOD samples. Using the median as a scoring function (vertical dashed lines) effectively separates InD from OOD.
• Figure 5: 2D L-shape toy dataset: The blue point cloud shows the training distribution and the red points are OOD test samples. The black contour represents the thresholded score function (median) separating InD from OOD samples. (a) Default KAN detector limited by the marginal distribution of $x_1$ and $x_2$. (b) Improved performance by concatenating the input features with the latent features of a variational autoencoder.