Are Neural Networks Collision Resistant?

TL;DR

This work analyzes collision resistance in a binary perceptron with a square-wave activation $\varphi_{\delta}$, introducing a regime where collisions exist but are algorithmically hard to find due to an overlap gap property (OGP). Using replica-symmetric and first-moment methods, it identifies a collision threshold $\alpha_c(q_1)$ and, in the strong hashing limit $\delta\to0$, a closed-form $\alpha_{OGP}^m(q_1)=\frac{\log 2+H_B(q_1)}{\log(1+m)}$, illustrating intrinsic computational hardness. The authors then propose cryptographic CRH constructions by composing the neural network with error-correcting codes under the Gilbert–Varshamov bound, yielding a compression ratio $\tilde{\alpha}=\alpha/r$ and a region where collision resistance can be guaranteed. They contrast their approach with Ajtai’s lattice-based function, highlighting differences in output space, reductions, and parameter regimes, and provide algorithmic simulations (rAMP) showing practical hardness well below the OGP threshold. Overall, the paper unveils a new source of hardness in neural-network–based primitives and offers a pathway to collision-resistant hashing via CC-coded SWP, with potential extensions to deeper architectures.

Abstract

When neural networks are trained to classify a dataset, one finds a set of weights from which the network produces a label for each data point. We study the algorithmic complexity of finding a collision in a single-layer neural net, where a collision is defined as two distinct sets of weights that assign the same labels to all data. For binary perceptrons with oscillating activation functions, we establish the emergence of an overlap gap property in the space of collisions. This is a topological property believed to be a barrier to the performance of efficient algorithms. The hardness is supported by numerical experiments using approximate message passing algorithms, for which the algorithms stop working well below the value predicted by our analysis. Neural networks provide a new category of candidate collision resistant functions, which for some parameter setting depart from constructions based on lattices. Beyond relevance to cryptography, our work uncovers new forms of computational hardness emerging in large neural networks which may be of independent interest.

Figures (12)

• Figure 1: Full lines: analytic prediction for the transition $\alpha_c$ above which no collision exists versus the internal overlap $q_1$, for different values of $\delta$. Dots: numerical estimates from exhaustive search.
• Figure 2: Typical behavior of $\phi^{\mathrm{a}}_m(p)$, for various values of $\alpha$, at $q_1=0.6$ and $m=5$. The activation is a square wave, with $\delta=0.3$. The onset value of the OGP is $\alpha=0.7$. For $\alpha>0.7$, an interval of external overlaps $p$ where $\phi<0$ is present.
• Figure 3: $\alpha_{OGP}^m$ versus the internal overlap $q_1$ for the Square Wave activation function, for various values of $\delta$, $m=5$.
• Figure 4: $\alpha_{\mathrm{OGP}}^m$ versus the internal overlap $q_1$ for the Square Wave activation function in the limit $\delta\to0$ for several values of $m$.
• Figure 5: The lowest compression rate of $NN\circ ECC$, i.e. $\alpha_{\mathrm{adv}}(q_{\mathrm{code}}) / r(q_{\mathrm{code}})$, as a function of $q_{\mathrm{code}}$, for different values of $\delta$. $\alpha_{\mathrm{adv}}(q_{\mathrm{code}})$ is given by a $m$-OGP computation with $m=5$.