Automating Variational Differentiation

TL;DR

This paper develops a novel framework for automating variational differentiation using the pullback of the $\mathbf{B}$ and $\mathbf{C}$ combinators, enabling analytic backpropagation and complex-number support without relying on traditional chain-rule AD. The CombDiff system symbolically differentiates and simplifies variational problems, converting gradients into human-readable forms and aligning results with efficient computational kernels. Key contributions include the $\mathbf{B}$- and $\mathbf{C}$-rule formulations, a complex-number extension via $\mathcal{V}$ and $\mathcal{W}$, and demonstrations on Quadratic forms, Conjugate Gradient, Hartree–Fock, and Maximally Localized Wannier Functions. This approach decouples differentiability from performance concerns, potentially improving readability and optimization in HPC and ab initio contexts, though equivalence-graph scaling with tensor symmetries remains a challenge.

Abstract

Many problems in Physics and Chemistry are formulated as the minimization of a functional. Therefore, methods for solving these problems typically require differentiating maps whose input and/or output are functions -- commonly referred to as variational differentiation. Such maps are not addressed at the mathematical level by the chain rule, which underlies modern symbolic and algorithmic differentiation (AD) systems. Although there are algorithmic solutions such as tracing and reverse accumulation, they do not provide human readability and introduce strict programming constraints that bottleneck performance, especially in high-performance computing (HPC) environments. In this manuscript, we propose a new computer theoretic model of differentiation by combining the pullback of the $\mathbf{B}$ and $\mathbf{C}$ combinators from the combinatory logic. Unlike frameworks based on the chain rule, this model differentiates a minimal complete basis for the space of computable functions. Consequently, the model is capable of analytic backpropagation and variational differentiation while supporting complex numbers. To demonstrate the generality of this approach we build a system named CombDiff, which can differentiate nontrivial variational problems such as Hartree-Fock (HF) theory and multilayer perceptrons.