MiniTensor: A Lightweight, High-Performance Tensor Operations Library

TL;DR

MiniTensor addresses the need for a lightweight, CPU-focused tensor library with automatic differentiation and neural network components. It introduces a three-layer architecture (Python API, PyO3 bindings, and a Rust execution engine) to deliver core tensor operations, dynamic autograd, and standard NN modules in a compact codebase. The work demonstrates a substantial footprint advantage over PyTorch and TensorFlow while preserving essential functionality for research and education on CPUs, and outlines a roadmap to broaden operator coverage and GPU support. Overall, MiniTensor offers an approachable, auditable alternative for prototyping and learning in environments where full-scale frameworks are impractical.

Abstract

We present MiniTensor, an open source tensor operations library that focuses on minimalism, correctness, and performance. MiniTensor exposes a familiar PyTorch-like Python API while it executes performance critical code in a Rust engine. The core supports dense $n$ dimensional tensors, broadcasting, reductions, matrix multiplication, reverse mode automatic differentiation, a compact set of neural network layers, and standard optimizers. In this paper, we describe the design of MiniTensor's architecture, including its efficient memory management, dynamic computation graph for gradients, and integration with Python via PyO3. We also compare the install footprint with PyTorch and TensorFlow to demonstrate that MiniTensor achieves a package size of only a few megabytes, several orders of magnitude smaller than mainstream frameworks, while preserving the essentials needed for research and development on CPUs. The repository can be found at https://github.com/neuralsorcerer/minitensor