Enabling Long FFT Convolutions on Memory-Constrained FPGAs via Chunking
Peter Wang, Neelesh Gupta, Viktor Prasanna
TL;DR
This work tackles long-context convolution on memory-constrained FPGAs by introducing a chunked FFT convolution approach with overlap-add reconstruction, enabling 450K length sequences convolved with 450K length filters on an Alveo U200 with only 2.8 MB BRAM. The method splits inputs into fixed-size chunks, processes them in the frequency domain using a radix-2 FFT kernel, and reconstructs the final output via overlap-add, achieving throughput that scales with chunk size while incurring only ~7% degradation for the longest sequences; the core computation dominates runtime, demonstrating feasibility for edge deployment of long-context primitives. The paper situates this work within Hyena-like architectures, showing FFT-based convolution is a key bottleneck for long contexts and providing a viable FPGA-based path for long-range modeling with explicit memory constraints. Significance lies in enabling memory-intensive, long-range context operations on edge devices, enabling efficient deployment of Hyena-like models without resorting to high-memory GPUs.
Abstract
The need for long-context reasoning has led to alternative neural network architectures besides Transformers and self-attention, a popular model being Hyena, which employs causal 1D-convolutions implemented with FFTs. Long convolutions enable efficient global context mixing, but requirements for intermediate results exceed the 2-3 MB Block RAM capacity of FPGAs. We present a chunked FFT convolution approach enabling 450K length sequence by 450K length filter convolutions on an Alveo U200 FPGA with 2.8 MB BRAM through chunking and overlap-add reconstruction. We find that throughput scales proportionally with chunk size while degrading minimally by 7% for our longest sequences, demonstrating that careful memory management enables deployment of long-context primitives on edge FPGAs without sacrificing performance.