Towards Understanding Learning Representations: To What Extent Do Different Neural Networks Learn the Same Representation

TL;DR

The paper tackles how similarly two identically-architected networks trained from different random initializations learn representations, proposing the neuron activation subspace match framework. It introduces the concepts of maximum match and simple matches, proves a Decomposition Theorem, and develops polynomial-time algorithms to compute matches and minimal components. Experimental results across CNNs on CIFAR10 and ImageNet reveal that representations in most convolutional layers exhibit surprisingly low subspace similarity, especially in deeper layers, prompting questions about metric validity and interpretability. The work provides a formal lens for comparing representations and sets the stage for deeper theoretical and empirical investigation into neural representations.

Abstract

It is widely believed that learning good representations is one of the main reasons for the success of deep neural networks. Although highly intuitive, there is a lack of theory and systematic approach quantitatively characterizing what representations do deep neural networks learn. In this work, we move a tiny step towards a theory and better understanding of the representations. Specifically, we study a simpler problem: How similar are the representations learned by two networks with identical architecture but trained from different initializations. We develop a rigorous theory based on the neuron activation subspace match model. The theory gives a complete characterization of the structure of neuron activation subspace matches, where the core concepts are maximum match and simple match which describe the overall and the finest similarity between sets of neurons in two networks respectively. We also propose efficient algorithms to find the maximum match and simple matches. Finally, we conduct extensive experiments using our algorithms. Experimental results suggest that, surprisingly, representations learned by the same convolutional layers of networks trained from different initializations are not as similar as prevalently expected, at least in terms of subspace match.

Figures (6)

• Figure 1: Maximal matching similarities of different architectures on different datasets under various $\epsilon$. The x-axis is along the propagation. (a) shows ResNet18 on CIFAR10 validation set, we leave other classical architectures like VGG in Supplementary material; (b) shows VGG16 on ImageNet validation set; (c) shows a deeper ResNet on CIFAR10.
• Figure 2: The distribution of the sizes of minimal matches of layers close to input and output respectively
• Figure 3: Maximum matching similarities of all the layers of different architectures under various $\epsilon$.
• Figure 4: Maximum match similarity between networks at different stages. Figure(a) shows the similarity of two untrained network. Figure(b) shows the similarity of the same network at different stages.
• Figure 5: Visualization of neurons in fc2. Each row includes top 9 images that maximize the activation of one neuron. The neurons in the same network are illustrated on the same side.

Theorems & Definitions (32)

• Definition 1: $\epsilon$-approximate match and exact match
• Lemma 2: Union-Close Lemma
• Definition 3: Maximum Match
• Definition 4: Simple Match
• Theorem 5: Decomposition Theorem
• Lemma 6: Intersection-Close Lemma
• Definition 7: $v$-Minimum Match
• Theorem 8
• Definition 9: $v$-Minimal Match
• Theorem 10