Neural Field Convolutions by Repeated Differentiation

TL;DR

This work addresses the challenge of performing general, large-scale convolutions directly in neural fields by introducing a convolution framework based on repeated differentiation of piecewise polynomial kernels. By representing kernels as sparse sets of Dirac deltas after differentiation and training a repeated integral field to provide the necessary antiderivatives, the authors achieve exact, efficient convolutions whose cost scales with the kernel’s Dirac count rather than its size. Key contributions include a principled kernel approximation strategy, a learnable repeated-integral representation, and extensive demonstrations across images, video, geometry, animation, and audio with spatially-varying kernels. The approach enables flexible, high-quality continuous filtering in neural-field representations, with practical implications for rendering, signal processing, and content-aware processing in continuous domains.

Abstract

Neural fields are evolving towards a general-purpose continuous representation for visual computing. Yet, despite their numerous appealing properties, they are hardly amenable to signal processing. As a remedy, we present a method to perform general continuous convolutions with general continuous signals such as neural fields. Observing that piecewise polynomial kernels reduce to a sparse set of Dirac deltas after repeated differentiation, we leverage convolution identities and train a repeated integral field to efficiently execute large-scale convolutions. We demonstrate our approach on a variety of data modalities and spatially-varying kernels.

Figures (19)

• Figure 1: The landscape of convolution methods as combinations of different operations applied to a signal and kernel (top), leading to a result (bottom).
• Figure 2: Overview of our approach. a) Given an arbitrary convolution kernel, we optimize for its piecewise polynomial approximation, which under repeated differentiation yields a sparse set of Dirac deltas. b) Given an original signal, we train a neural field that captures the repeated integral of the signal. c) The continuous convolution of the original signal and the convolution kernel is obtained by a discrete convolution of the sparse Dirac deltas from a) and corresponding sparse samples of the neural integral field from b).
• Figure 3: Repeated differentiation of a bilinear patch ($\TextOrMath{$d$\xspace}{d}=\TextOrMath{$n$\xspace}{n}=2$). After the first differentiation, the patch exhibits linear variation only along the vertical dimension. After subsequent differentiations, we obtain a constant patch, two vertical lines, and, finally, four Dirac deltas.
• Figure 4: Kernel representation in 1D for the case $\TextOrMath{$n$\xspace}{n}=2$, i.e., a piecewise linear function. Top row: The original continuous kernel $g$g has a continuous second derivative. Bottom row: We approximate $g$g with a piecewise linear function $\hat{\TextOrMath{$g$\xspace}{g}}$, which reduces to a sparse set of Dirac deltas in its second derivative.
• Figure 5: 1D ramps of different orders (redrawn from heckbert1986filtering). Each ramp is the antiderivative of its predecessor.