A Note on Adjoint Linear Algebra
Uwe Naumann
TL;DR
This work presents a novel, AD-based derivation of adjoint linear systems by leveraging adjoint BLAS, deriving adjoints for scalar multiplication, inner products, and matrix operations, and culminating in an alternative, consistent proof for the adjoint of $A\mathbf{x}=\mathbf{b}$. It emphasizes performing computations at the highest BLAS level and demonstrates how a single factorization can be reused to efficiently obtain tangent and adjoint solutions, potentially reducing costs from $O(n^3)$ to $O(n^2)$. The results extend naturally to higher-order tangents and adjoints, enabling efficient higher-order AD for linear systems. The approach integrates AD theory with practical, high-performance linear algebra workflows.
Abstract
A new proof for adjoint systems of linear equations is presented. The argument is built on the principles of Algorithmic Differentiation. Application to scalar multiplication sets the base line. Generalization yields adjoint inner vector, matrix-vector, and matrix-matrix products leading to an alternative proof for first- as well as higher-order adjoint linear systems.