Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts

Hyunsu Kim, Yegon Kim, Hongseok Yang, Juho Lee

TL;DR

This work tackles the rigidity of fixed symmetry in Group Equivariant CNNs by proposing Variational Partial G-CNN (VP G-CNN), which learns data-dependent partial equivariance through a variationally trained, input-conditioned distribution $q(u|f)$. It supports both continuous rotations with $SO(2)$ and discrete color shifts with $H_m$, using a reparameterizable sampling scheme to stabilize training and an ELBO objective that balances classification performance and KL regularization toward full equivariance. The approach applies input-aware partial convolutions to select layers, aided by a lightweight encoder to predict distribution parameters, and achieves robust accuracy and improved uncertainty calibration across MNIST67-180, CIFAR10, ColorMNIST, and Flowers102. The results demonstrate meaningful per-instance/per-class variation in required equivariance and point toward broader applicability to other symmetry groups in real-world data.

Abstract

Group Equivariant CNNs (G-CNNs) have shown promising efficacy in various tasks, owing to their ability to capture hierarchical features in an equivariant manner. However, their equivariance is fixed to the symmetry of the whole group, limiting adaptability to diverse partial symmetries in real-world datasets, such as limited rotation symmetry of handwritten digit images and limited color-shift symmetry of flower images. Recent efforts address this limitation, one example being Partial G-CNN which restricts the output group space of convolution layers to break full equivariance. However, such an approach still fails to adjust equivariance levels across data. In this paper, we propose a novel approach, Variational Partial G-CNN (VP G-CNN), to capture varying levels of partial equivariance specific to each data instance. VP G-CNN redesigns the distribution of the output group elements to be conditioned on input data, leveraging variational inference to avoid overfitting. This enables the model to adjust its equivariance levels according to the needs of individual data points. Additionally, we address training instability inherent in discrete group equivariance models by redesigning the reparametrizable distribution. We demonstrate the effectiveness of VP G-CNN on both toy and real-world datasets, including MNIST67-180, CIFAR10, ColorMNIST, and Flowers102. Our results show robust performance, even in uncertainty metrics.

Variational Partial Group Convolutions for Input-Aware Partial Equivariance of Rotations and Color-Shifts

TL;DR

This work tackles the rigidity of fixed symmetry in Group Equivariant CNNs by proposing Variational Partial G-CNN (VP G-CNN), which learns data-dependent partial equivariance through a variationally trained, input-conditioned distribution . It supports both continuous rotations with and discrete color shifts with , using a reparameterizable sampling scheme to stabilize training and an ELBO objective that balances classification performance and KL regularization toward full equivariance. The approach applies input-aware partial convolutions to select layers, aided by a lightweight encoder to predict distribution parameters, and achieves robust accuracy and improved uncertainty calibration across MNIST67-180, CIFAR10, ColorMNIST, and Flowers102. The results demonstrate meaningful per-instance/per-class variation in required equivariance and point toward broader applicability to other symmetry groups in real-world data.

Abstract

Group Equivariant CNNs (G-CNNs) have shown promising efficacy in various tasks, owing to their ability to capture hierarchical features in an equivariant manner. However, their equivariance is fixed to the symmetry of the whole group, limiting adaptability to diverse partial symmetries in real-world datasets, such as limited rotation symmetry of handwritten digit images and limited color-shift symmetry of flower images. Recent efforts address this limitation, one example being Partial G-CNN which restricts the output group space of convolution layers to break full equivariance. However, such an approach still fails to adjust equivariance levels across data. In this paper, we propose a novel approach, Variational Partial G-CNN (VP G-CNN), to capture varying levels of partial equivariance specific to each data instance. VP G-CNN redesigns the distribution of the output group elements to be conditioned on input data, leveraging variational inference to avoid overfitting. This enables the model to adjust its equivariance levels according to the needs of individual data points. Additionally, we address training instability inherent in discrete group equivariance models by redesigning the reparametrizable distribution. We demonstrate the effectiveness of VP G-CNN on both toy and real-world datasets, including MNIST67-180, CIFAR10, ColorMNIST, and Flowers102. Our results show robust performance, even in uncertainty metrics.
Paper Structure (39 sections, 2 theorems, 25 equations, 9 figures, 7 tables)

This paper contains 39 sections, 2 theorems, 25 equations, 9 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Group Equivariance and Partial Equivariance
  4. G-CNN and Partial G-CNN
  5. Lifting convolution.
  6. Group convolution.
  7. Partial group convolution.
  8. Color equivariance $H_m$.
  9. Variational Partial G-CNN
  10. Input-Aware Partial Convolution
  11. Variational Inference of $q(u|f)$
  12. Rotation $SO(2)$ (continuous).
  13. Color-shift $H_m$ (discrete).
  14. Implementation
  15. Related Work
  16. ...and 24 more sections

Key Result

Proposition 3.1

Assume that the conditional distribution $q(u|f)$ is partially equivariant with respect to a group $G$ and an equivariant subset $C\subseteq{\mathcal{F}}$ in the following sense: where ${\mathcal{L}}_gf(u) = f(g^{-1}u)$, and kernel $k$ and input $f$ of the group convolutions defined in eq:bayes-partial are bounded. Then, the group convolutions are also partially equivariant to $G$ and $C$.

Figures (9)

  • Figure 1: Illustrative example of partial equivariance. (a) $180^\circ$-rotation of 6 is regarded as 9 but 7 is not. (b) The color-shifted image of Barberton Daisy looks similar to Osteospermum.
  • Figure 2: Architecture of Variational Partial Group Convolutions. The colored boxes are the features at each layer and the white boxes are zero features removed out by the distribution $q(u|f)$, where $u=r_\phi(f,\epsilon)$.
  • Figure 3: As the $L_\mathrm{KLD}$ increases, the distribution $p(u|f)$ expands, but upon reaching a certain point where $L_\mathrm{CLS}$ is affected, the distribution becomes constrained.
  • Figure 4: Partial equivariance trained on MNIST67-180. The x-axis represents the rotation angle of the input and the y-axis represents the model's confidence for the corresponding class. The model exhibits equivariance to rotations on semi-circle for image 6, whereas it shows full equivariance for image 7.
  • Figure 5: The x-axis represents the magnitude of the shift in the Hue space of the input, while the y-axis represents the model's confidence for the corresponding class. The image at zero hue- shift represents the original image. (a) Barberton Daisy exhibits partial equivariance because a shift of -0.17 or 0.17 overlaps with other flowers, while (b) Snapdragon demonstrates full equivariance owing to its distinctive appearance.
  • ...and 4 more figures

Theorems & Definitions (4)

  • Definition 2.1: $(S, \varepsilon, G)$-Partial Equivariance
  • Definition 2.2: $(C, \varepsilon, G)$-Partial Equivariance on Feature Map
  • Proposition 3.1
  • Proposition 3.2