Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CliffordNet: All You Need is Geometric Algebra

Zhongping Ji

TL;DR

CliffordNet introduces a geometry-centric vision backbone that replaces traditional FFN-heavy mixing with a Clifford Geometric Product-based interaction, achieving linear-time complexity via a sparse shifted mechanism. By modeling feature evolution as a local, geometry-driven diffusion-rotation process and instantiating the context with a factorized discrete Laplacian, the network preserves 2D topology and delivers strong parameter efficiency. Empirical results on CIFAR-100 show that even the No-FFN variant attains competitive accuracy, with the Base model reaching 78.05% while using under 4M parameters, signaling a potential paradigm shift where algebraic completeness substitutes for heavy channel mixing. The work also discusses theoretical interpretations through geometric reaction-diffusion dynamics and hardware-aware implementations, outlining clear paths for scaling and optimization in larger-scale vision tasks.

Abstract

Modern computer vision architectures, from CNNs to Transformers, predominantly rely on the stacking of heuristic modules: spatial mixers (Attention/Conv) followed by channel mixers (FFNs). In this work, we challenge this paradigm by returning to mathematical first principles. We propose the \textbf{Clifford Algebra Network (CAN)}, also referred to as CliffordNet, a vision backbone grounded purely in Geometric Algebra. Instead of engineering separate modules for mixing and memory, we derive a unified interaction mechanism based on the \textbf{Clifford Geometric Product} ($uv = u \cdot v + u \wedge v$). This operation ensures algebraic completeness regarding the Geometric Product by simultaneously capturing feature coherence (via the generalized inner product) and structural variation (via the exterior wedge product). Implemented via an efficient sparse rolling mechanism with \textbf{strict linear complexity $\mathcal{O}(N)$}, our model reveals a surprising emergent property: the geometric interaction is so representationally dense that standard Feed-Forward Networks (FFNs) become redundant. Empirically, CliffordNet establishes a new Pareto frontier: our \textbf{Nano} variant achieves \textbf{76.41\%} accuracy on CIFAR-100 with only \textbf{1.4M} parameters, effectively matching the heavy-weight ResNet-18 (11.2M) with \textbf{$8\times$ fewer parameters}, while our \textbf{Base} variant sets a new SOTA for tiny models at \textbf{78.05\%}. Our results suggest that global understanding can emerge solely from rigorous, algebraically complete local interactions, potentially signaling a shift where \textit{geometry is all you need}. Code is available at https://github.com/ParaMind2025/CAN.

CliffordNet: All You Need is Geometric Algebra

TL;DR

CliffordNet introduces a geometry-centric vision backbone that replaces traditional FFN-heavy mixing with a Clifford Geometric Product-based interaction, achieving linear-time complexity via a sparse shifted mechanism. By modeling feature evolution as a local, geometry-driven diffusion-rotation process and instantiating the context with a factorized discrete Laplacian, the network preserves 2D topology and delivers strong parameter efficiency. Empirical results on CIFAR-100 show that even the No-FFN variant attains competitive accuracy, with the Base model reaching 78.05% while using under 4M parameters, signaling a potential paradigm shift where algebraic completeness substitutes for heavy channel mixing. The work also discusses theoretical interpretations through geometric reaction-diffusion dynamics and hardware-aware implementations, outlining clear paths for scaling and optimization in larger-scale vision tasks.

Abstract

Modern computer vision architectures, from CNNs to Transformers, predominantly rely on the stacking of heuristic modules: spatial mixers (Attention/Conv) followed by channel mixers (FFNs). In this work, we challenge this paradigm by returning to mathematical first principles. We propose the \textbf{Clifford Algebra Network (CAN)}, also referred to as CliffordNet, a vision backbone grounded purely in Geometric Algebra. Instead of engineering separate modules for mixing and memory, we derive a unified interaction mechanism based on the \textbf{Clifford Geometric Product} (). This operation ensures algebraic completeness regarding the Geometric Product by simultaneously capturing feature coherence (via the generalized inner product) and structural variation (via the exterior wedge product). Implemented via an efficient sparse rolling mechanism with \textbf{strict linear complexity }, our model reveals a surprising emergent property: the geometric interaction is so representationally dense that standard Feed-Forward Networks (FFNs) become redundant. Empirically, CliffordNet establishes a new Pareto frontier: our \textbf{Nano} variant achieves \textbf{76.41\%} accuracy on CIFAR-100 with only \textbf{1.4M} parameters, effectively matching the heavy-weight ResNet-18 (11.2M) with \textbf{ fewer parameters}, while our \textbf{Base} variant sets a new SOTA for tiny models at \textbf{78.05\%}. Our results suggest that global understanding can emerge solely from rigorous, algebraically complete local interactions, potentially signaling a shift where \textit{geometry is all you need}. Code is available at https://github.com/ParaMind2025/CAN.
Paper Structure (17 sections, 9 equations, 1 figure, 4 tables, 1 algorithm)

This paper contains 17 sections, 9 equations, 1 figure, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preliminaries
  5. A General Framework for Feature Evolution
  6. The Clifford Interaction Ansatz
  7. Context Instantiation via Discrete Laplacian
  8. Efficient Realization: The Shifted Geometric Product
  9. Gated Geometric Residual (GGR)
  10. Architectural Variants
  11. Experiments
  12. Main Results and Efficiency Analysis
  13. Discussion
  14. Why does the "No-FFN" Paradigm Work?
  15. Theoretical View: A Geometric Reaction-Diffusion System
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Visualizing the Efficient Wedge Product. We operationalize the Geometric Product via a linear-complexity rolling mechanism. The Wedge Product branch captures structural variation. Instead of computing a full quadratic tensor, we cyclically shift the feature streams ($\mathcal{T}_s$) to compute cross-term differences. Geometrically, this constructs a bivector (green plane), representing the oriented area and orthogonality between the state $\mathbf{h}$ and context $\mathbf{c}$.