The algebra and the geometry aspect of Deep learning
Tsemo Aristide
TL;DR
This work surveys a mathematical foundation for deep learning by linking algebraic and geometric perspectives with classic optimization and neural architectures. It develops a coordinate-free view of backpropagation using the canonical matrix scalar product, connects learning to manifolds and algebraic varieties, and traverses from the perceptron to the Transformer with explicit forward and backward mappings. The text provides concrete algorithmic details for ReLU, softmax, initialization, cross-entropy loss, and backpropagation, alongside a Transformer attention framework and its NLP training workflow. The contributions aim to unify deep learning practice with rigorous mathematical structures and propose a mathematical language model concept, supported by implementation resources.
Abstract
This paper investigates the foundations of deep learning through insight of geometry, algebra and differential calculus. At is core, artificial intelligence relies on assumption that data and its intrinsic structure can be embedded into vector spaces allowing for analysis through geometric and algebraic methods. We thrace the development of neural networks from the perceptron to the transformer architecture, emphasizing on the underlying geometric structures and differential processes that govern their behavior. Our original approach highlights how the canonical scalar product on matrix spaces naturally leads to backpropagation equations yielding to a coordinate free formulation. We explore how classification problems can reinterpreted using tools from differential and algebraic geometry suggesting that manifold structure, degree of variety, homology may inform both convergence and interpretability of learning algorithms We further examine how neural networks can be interpreted via their associated directed graph, drawing connection to a Quillen model defined in [1] and [13] to describe memory as an homotopy theoretic property of the associated network.