Robust online reconstruction of continuous-time signals from a lean spike train ensemble code

TL;DR

This work introduces a deterministic framework to encode continuous-time signals into lean spike trains via a convolve-then-threshold mechanism across kernel ensembles, and provides a linear-inversion solution in the Hilbert space of shifted kernels for reconstruction. It proves a Perfect Reconstruction Theorem for signals in a finite-innovation class and derives a robust, approximate-reconstruction bound under model deviations, along with a windowed online decoding algorithm whose convergence to the optimal solution is guaranteed under realistic conditions. The approach includes a stability analysis of the Gram matrix and a practical windowed scheme to control conditioning and enable real-time decoding. Empirical validation on large-scale audio data shows high reconstruction accuracy at a low spike rate (around 1/5 Nyquist) and favorable comparisons to state-of-the-art sparse coding methods, with favorable runtime characteristics.

Abstract

Sensory stimuli in animals are encoded into spike trains by neurons, offering advantages such as sparsity, energy efficiency, and high temporal resolution. This paper presents a signal processing framework that deterministically encodes continuous-time signals into biologically feasible spike trains, and addresses the questions about representable signal classes and reconstruction bounds. The framework considers encoding of a signal through spike trains generated by an ensemble of neurons using a convolve-then-threshold mechanism with various convolution kernels. A closed-form solution to the inverse problem, from spike trains to signal reconstruction, is derived in the Hilbert space of shifted kernel functions, ensuring sparse representation of a generalized Finite Rate of Innovation (FRI) class of signals. Additionally, inspired by real-time processing in biological systems, an efficient iterative version of the optimal reconstruction is formulated that considers only a finite window of past spikes, ensuring robustness of the technique to ill-conditioned encoding; convergence guarantees of the windowed reconstruction to the optimal solution are then provided. Experiments on a large audio dataset demonstrate excellent reconstruction accuracy at spike rates as low as one-fifth of the Nyquist rate, while showing clear competitive advantage in comparison to state-of-the-art sparse coding techniques in the low spike rate regime.

Figures (7)

• Figure 1: The convolve and threshold mechanism described in the coding model for a single kernel. Top: a sample signal (in blue) is shown overlayed with a convolution kernel (in red). Below: the result of convolution in blue and the threshold function for the kernel in green. Spikes times are marked at the threshold crossing level with red dots.
• Figure 2: Illustration of Lemma \ref{['windowLemma']}. The diagram shows the convergence of the windowed orthogonal complement $\phi^{\perp}_{n+1,w}$ of spike $\phi_{n+1}$ to $\phi^{\perp}_{n+1}$ by orthogonalizing $\phi_{n+1}$ across the partitions of spikes. (a) Displays all spikes up to $\phi_{n+1}$ (black), with partitions circled: $v_1$ (green), $v_k$ (red), and $v_{k+1}$ (blue). Spikes contained in each partition are shaded accordingly, with the time of each spike marked by a purple dot. (b), (c), and (d) show orthogonal complements $\phi^{\perp}_{n+1,v_1}$, $\phi^{\perp}_{n+1,v_k}$, and $\phi^{\perp}_{n+1,v_{k+1}}$ respectively. The support of $\phi^{\perp}_{n+1,w}$ extends as more partitions are included, with the extending tail for each additional partition highlighted in red. This tail's diminishing energy as more partitions are added illustrates Lemma \ref{['windowLemma']}.
• Figure 3: Figure illustrating the vector projections of the input signal $X$ onto vectors $\phi^{\perp}_{n+1}$ and $\phi^{\perp}_{n+1,w}$. The red vector represents $\Tilde{X}$, the projection of $X$ within the plane formed by $\phi^{\perp}_{n+1}$ and $\phi^{\perp}_{n+1,w}$. The vectors $\phi^{\perp}_{n+1}$ and $\phi^{\perp}_{n+1,w}$, as well as the projections $X_u$ and $X_v$ of $\Tilde{X}$ onto them, are indicated in blue. The vector $p_w$, representing the difference between $\phi^{\perp}_{n+1,w}$ and $\phi^{\perp}_{n+1}$, is also shown in blue. The angles $a$ between $\Tilde{X}$ and $X_u$, $b$ between $\Tilde{X}$ and $X_v$, and $a-b$ between $\phi^{\perp}_{n+1,w}$ and $\phi^{\perp}_{n+1}$ are marked.
• Figure 4: The scenario illustrating the need for Assumption \ref{['assumption2']}. See text for details. For derivation of $||\mathcal{P}_{\mathcal{S}(\bigcup_1^{N}\{\phi_i\} \setminus \{\phi_n\})}(\phi_n)|| \rightarrow 1$ see supplementary_key.
• Figure 5: Comprehensive results of experiments on 600 audio snippets with 50 kernels. (a) Scatter plot of reconstructions where each dot represents a single reconstruction performed on one of the 600 sound snippet for a particular setting of the ahp parameters as described in Section \ref{['comprehensiveExpt']}. The plot shows the SNR value of the reconstructions (x-axis) against corresponding spike-rate of the ensemble (y-axis). The trend line in purple is generated using seaborn regression fit, and the black dot on the line highlights the point on the trend line at 20 dB SNR which has an average spike-rate of approximately one-fifth the Nyquist rate. (b) Bar graph showing the distribution of all reconstructions from (a) at different SNRs. Reconstructions are binned at intervals of 3dB in the range 0 to 30 dB SNR (x-axis) and the bars show the percentage of all reconstructions that fall within the corresponding bin (y-axis). As is evident, the maximum fraction of reconstructions falls in the bin of 18-21 dB. (c) Strip plot showing the distribution of spike rates across all the bins from (b). Like (b), the x-axis shows the SNR values binned at intervals of 3dB and the y-axis shows the spike rates of reconstructions as a fraction of the Nyquist rate, 44.1kHz. The blue line, passing through the strip plot, connects the average spike rates of reconstructions in each bin (averages of each bin are shown by blue markers).

Theorems & Definitions (15)

• Corollary 1
• Corollary 2
• proof
• Corollary 3
• Theorem 1
• Lemma 1
• Lemma 2
• Theorem 2
• Theorem 3: Condition Number Theorem
• Lemma 3