Towards a General Purpose CNN for Long Range Dependencies in $N$D

TL;DR

The paper tackles the need for a general-purpose CNN that handles varying data lengths, resolutions, and dimensionalities. It introduces CCNN, which uses a kernel generator network to produce continuous convolutional kernels, enabling resolution- and dimension-agnostic processing and modeling long-range dependencies at every layer. Architectural improvements—such as depthwise separable continuous convolutions and principled kernel initialization—are validated on 1D and 2D benchmarks, achieving state-of-the-art results on several tasks with fewer parameters. This approach simplifies deployment across data types and scales while preserving global context, with notable applicability to irregularly-sampled data and dynamic resolutions.

Abstract

The use of Convolutional Neural Networks (CNNs) is widespread in Deep Learning due to a range of desirable model properties which result in an efficient and effective machine learning framework. However, performant CNN architectures must be tailored to specific tasks in order to incorporate considerations such as the input length, resolution, and dimentionality. In this work, we overcome the need for problem-specific CNN architectures with our Continuous Convolutional Neural Network (CCNN): a single CNN architecture equipped with continuous convolutional kernels that can be used for tasks on data of arbitrary resolution, dimensionality and length without structural changes. Continuous convolutional kernels model long range dependencies at every layer, and remove the need for downsampling layers and task-dependent depths needed in current CNN architectures. We show the generality of our approach by applying the same CCNN to a wide set of tasks on sequential (1$\mathrm{D}$) and visual data (2$\mathrm{D}$). Our CCNN performs competitively and often outperforms the current state-of-the-art across all tasks considered.

Figures (4)

• Figure 1: Continuous convolutional kernels. A continuous convolutional kernel is parameterized with a small neural network $\mathrm{G}_{\mathrm{Kernel}}$ that receives coordinates $\mathbf{{c}}_i \in {\mathbb{R}}^{\mathrm{D}}$ as input and outputs the value of the convolutional kernel at that position ${\mathbf{{K}}}(\mathbf{{c}}_i) {=} \mathrm{G}_{\mathrm{Kernel}}(\mathbf{{c}}_i) \in {\mathbb{R}}^{\mathrm{N_{in}}\times\mathrm{N_{out}}}$ (\ref{['fig:ckconv_mlp']}). The continuous parameterization of ${\mathbf{{K}}}$ allows the convolutional layer to (i) model long range dependencies, (ii) handle irregularly sampled data, and (iii) be used across different resolutions. Additionally, changing the dimensionality of the coordinates $\mathbf{{c}}_i$ can be used to construct convolutional kernels for sequential (\ref{['fig:ckconv_1d']}), visual (\ref{['fig:ckconv_2d']}), and higher dimensional data (\ref{['fig:ckconv_3d']}) with the same kernel generator network.
• Figure 2: The Continuous CNN architecture.
• Figure 3: 1D vs. 2D image classification.
• Figure 4: Positive and negative samples from the Path-X dataset