Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramian

Patricia Pauli, Ruigang Wang, Ian R. Manchester, Frank Allgöwer

TL;DR

This work addresses end-to-end robustness of 1D CNNs by enforcing a predefined $\rho$-Lipschitz bound through a direct, layer-wise parameterization. It combines the Cayley transform to parameterize orthogonal-like components with a controllability Gramian-based approach to satisfy LMIs, enabling unconstrained training of Lipschitz-bounded networks. The method is validated on MIT-BIH ECG classification, where LipCNNs maintain competitive accuracy while enhancing robustness to adversarial perturbations. The framework naturally handles pooling and is extendable to 2D CNNs via a 2D systems formulation, offering a scalable path to provably robust CNNs.

Abstract

We establish a layer-wise parameterization for 1D convolutional neural networks (CNNs) with built-in end-to-end robustness guarantees. In doing so, we use the Lipschitz constant of the input-output mapping characterized by a CNN as a robustness measure. We base our parameterization on the Cayley transform that parameterizes orthogonal matrices and the controllability Gramian of the state space representation of the convolutional layers. The proposed parameterization by design fulfills linear matrix inequalities that are sufficient for Lipschitz continuity of the CNN, which further enables unconstrained training of Lipschitz-bounded 1D CNNs. Finally, we train Lipschitz-bounded 1D CNNs for the classification of heart arrythmia data and show their improved robustness.

Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramian

TL;DR

This work addresses end-to-end robustness of 1D CNNs by enforcing a predefined -Lipschitz bound through a direct, layer-wise parameterization. It combines the Cayley transform to parameterize orthogonal-like components with a controllability Gramian-based approach to satisfy LMIs, enabling unconstrained training of Lipschitz-bounded networks. The method is validated on MIT-BIH ECG classification, where LipCNNs maintain competitive accuracy while enhancing robustness to adversarial perturbations. The framework naturally handles pooling and is extendable to 2D CNNs via a 2D systems formulation, offering a scalable path to provably robust CNNs.

Abstract

We establish a layer-wise parameterization for 1D convolutional neural networks (CNNs) with built-in end-to-end robustness guarantees. In doing so, we use the Lipschitz constant of the input-output mapping characterized by a CNN as a robustness measure. We base our parameterization on the Cayley transform that parameterizes orthogonal matrices and the controllability Gramian of the state space representation of the convolutional layers. The proposed parameterization by design fulfills linear matrix inequalities that are sufficient for Lipschitz continuity of the CNN, which further enables unconstrained training of Lipschitz-bounded 1D CNNs. Finally, we train Lipschitz-bounded 1D CNNs for the classification of heart arrythmia data and show their improved robustness.
Paper Structure (14 sections, 34 equations, 3 figures, 1 table)

This paper contains 14 sections, 34 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Preliminaries
  4. State space representation for convolutions
  5. Lipschitz constant estimation
  6. Cayley transform
  7. Direct parameterization
  8. Fully connected layers
  9. Parameterization by the controllability Gramian
  10. Convolutional layers
  11. Maximum pooling layers
  12. $\rho$-Lipschitz layers
  13. Simulation results
  14. Conclusion

Figures (3)

  • Figure 1: Different heart wave classes.
  • Figure 2: (a) CNN used for ECG dataset. (b) Vanilla, L2 regularized ($\gamma= 0.1$) and Lipschitz ($\rho = 10, 50$) CNN on adversarial examples with different perturbation strengths $\epsilon$ created using the L2 projected gradient descent attack madry2017towards.
  • Figure 3: Accuracy robustness tradeoff. Test accuracy over Lipschitz constant from SDP pauli2023lipschitz, averaged over 5 CNNs.

Theorems & Definitions (4)

  • proof
  • proof
  • proof
  • proof