Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Null Space Properties of Neural Networks with Applications to Image Steganography

Xiang Li, Kevin M. Short

TL;DR

This work extends the classical linear null space concept to nonlinear maps by defining the nonlinear null space $N(f)$ for neural networks and examining its implications for FCNNs and CNNs. It identifies practical constructs such as partial null spaces $PN_{f_1}(f)$ and shows how the first linear layer often captures the overall null space, with ReLU networks providing intuitive geometric illustrations. Leveraging $N(f)$, the authors propose a steganography method that hides a secret image inside a cover image by projecting onto $N(f)$ and combining with the orthogonal component, ensuring $f(S)=f(H)$ while $S$ visually resembles $C$. Through experiments on MNIST, FMNIST, EMNIST, and CIFAR-10, the paper demonstrates that null-space steganography can reliably induce the hidden content’s classification while preserving cover-looking visuals, and it discusses broader reliability concerns about how NNs “see” data compared to humans.

Abstract

This paper explores the null space properties of neural networks. We extend the null space definition from linear to nonlinear maps and discuss the presence of a null space in neural networks. The null space of a given neural network can tell us the part of the input data that makes no contribution to the final prediction so that we can use it to trick the neural network. This reveals an inherent weakness in neural networks that can be exploited. One application described here leads to a method of image steganography. Through experiments on image datasets such as MNIST, we show that we can use null space components to force the neural network to choose a selected hidden image class, even though the overall image can be made to look like a completely different image. We conclude by showing comparisons between what a human viewer would see, and the part of the image that the neural network is actually using to make predictions and, hence, show that what the neural network ``sees'' is completely different than what we would expect.

Null Space Properties of Neural Networks with Applications to Image Steganography

TL;DR

This work extends the classical linear null space concept to nonlinear maps by defining the nonlinear null space for neural networks and examining its implications for FCNNs and CNNs. It identifies practical constructs such as partial null spaces and shows how the first linear layer often captures the overall null space, with ReLU networks providing intuitive geometric illustrations. Leveraging , the authors propose a steganography method that hides a secret image inside a cover image by projecting onto and combining with the orthogonal component, ensuring while visually resembles . Through experiments on MNIST, FMNIST, EMNIST, and CIFAR-10, the paper demonstrates that null-space steganography can reliably induce the hidden content’s classification while preserving cover-looking visuals, and it discusses broader reliability concerns about how NNs “see” data compared to humans.

Abstract

This paper explores the null space properties of neural networks. We extend the null space definition from linear to nonlinear maps and discuss the presence of a null space in neural networks. The null space of a given neural network can tell us the part of the input data that makes no contribution to the final prediction so that we can use it to trick the neural network. This reveals an inherent weakness in neural networks that can be exploited. One application described here leads to a method of image steganography. Through experiments on image datasets such as MNIST, we show that we can use null space components to force the neural network to choose a selected hidden image class, even though the overall image can be made to look like a completely different image. We conclude by showing comparisons between what a human viewer would see, and the part of the image that the neural network is actually using to make predictions and, hence, show that what the neural network ``sees'' is completely different than what we would expect.
Paper Structure (17 sections, 7 theorems, 6 equations, 14 figures)

This paper contains 17 sections, 7 theorems, 6 equations, 14 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Null space of linear and nonlinear maps
  4. Null space of fully connected neural networks
  5. Null space of convolutional neural network
  6. Photo steganography based on the null space of a neural network
  7. Experimental results
  8. Hide the digits
  9. Results on other datasets
  10. Discussions and conclusions
  11. What we see and what NN sees:
  12. Null space of NNs and reliability issue:
  13. Proofs and additional results from Section \ref{['sect:nullspace']}
  14. Properties of the null space in nonlinear maps
  15. Proofs of lemmas from Section \ref{['sect:nullspace']}
  16. ...and 2 more sections

Key Result

Lemma 2.3

Let $f:\mathbb{R}^n\rightarrow \mathbb{R}^m$ be a nonlinear map. Every partial null space $PN(f)$ is a subspace of $N(f)$, $\dim PN(f)\leq \dim N(f)$.

Figures (14)

  • Figure 1: The surface plot of a (2,1,3,1)-ReLU NN.
  • Figure 2: An example of creating a steganographic image with null space of a ReLU NN.
  • Figure 3: Examples of steganographic images with MNIST dataset.
  • Figure 4: More examples on MNIST dataset.
  • Figure 5: Examples of steganographic images with FMNIST dataset.
  • ...and 9 more figures

Theorems & Definitions (16)

  • Definition 2.1: Null space of a nonlinear map
  • Definition 2.2: Partial null space of a map
  • Lemma 2.3
  • Lemma 2.4
  • Corollary 2.5
  • Definition 2.6
  • Proposition A.1
  • proof
  • Proposition A.2
  • proof
  • ...and 6 more