Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Matrix-Free Jacobian Chaining

Uwe Naumann

TL;DR

This work reframes Jacobian accumulation for complex, modular simulations as a Matrix-Free Jacobian Chain Product (MFJCP) where only tangents and adjoints are available. It establishes a formal model using chain rules, DAGs, and tape memory, and shows that the general problem is NP-complete, while a dynamic programming approach yields polynomial-time solutions for dense, structured cases. The Limited-Memory variant introduces feasibility under tape constraints, enabling optimal bracketing under realistic memory limits. A public reference implementation demonstrates substantial performance gains over naive homogeneous tangent/adjoint strategies, and a tunnel flow case study with OpenFOAM illustrates practical applicability. The work points to future directions in sparsity exploitation and DAG-based extensions to broader matrix DAGs, with potential for significant reductions in computational cost in large-scale simulations.

Abstract

The efficient computation of Jacobians represents a fundamental challenge in computational science and engineering. Large-scale modular numerical simulation programs can be regarded as sequences of evaluations of in our case differentiable subprograms with corresponding elemental Jacobians. The latter are typically not available. Tangent and adjoint versions of the individual subprograms are assumed to be given as results of algorithmic differentiation instead. The classical (Jacobian) Matrix Chain Product problem is reformulated in terms of matrix-free Jacobian-matrix (tangents) and matrix-Jacobian products (adjoints), subject to limited memory for storing information required by latter. All numerical results can be reproduced using an open-source reference implementation.

Matrix-Free Jacobian Chaining

TL;DR

This work reframes Jacobian accumulation for complex, modular simulations as a Matrix-Free Jacobian Chain Product (MFJCP) where only tangents and adjoints are available. It establishes a formal model using chain rules, DAGs, and tape memory, and shows that the general problem is NP-complete, while a dynamic programming approach yields polynomial-time solutions for dense, structured cases. The Limited-Memory variant introduces feasibility under tape constraints, enabling optimal bracketing under realistic memory limits. A public reference implementation demonstrates substantial performance gains over naive homogeneous tangent/adjoint strategies, and a tunnel flow case study with OpenFOAM illustrates practical applicability. The work points to future directions in sparsity exploitation and DAG-based extensions to broader matrix DAGs, with potential for significant reductions in computational cost in large-scale simulations.

Abstract

The efficient computation of Jacobians represents a fundamental challenge in computational science and engineering. Large-scale modular numerical simulation programs can be regarded as sequences of evaluations of in our case differentiable subprograms with corresponding elemental Jacobians. The latter are typically not available. Tangent and adjoint versions of the individual subprograms are assumed to be given as results of algorithmic differentiation instead. The classical (Jacobian) Matrix Chain Product problem is reformulated in terms of matrix-free Jacobian-matrix (tangents) and matrix-Jacobian products (adjoints), subject to limited memory for storing information required by latter. All numerical results can be reproduced using an open-source reference implementation.
Paper Structure (14 sections, 2 theorems, 26 equations, 2 figures, 1 table)

This paper contains 14 sections, 2 theorems, 26 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Matrix-Free Jacobian Chain Product
  3. Matrix-Free Dense Jacobian Chain Product Bracketing
  4. Example.
  5. Example.
  6. Limited-Memory Matrix-Free Dense Jaco- bian Chain Product Bracketing
  7. Example.
  8. Implementation and Results
  9. Example.
  10. Case Study
  11. Conclusion and Outlook
  12. Exploitation of Jacobian Sparsity.
  13. Exploitation of DAG Structure.
  14. From Matrix Chains to Matrix DAGs.

Key Result

Theorem 3.1

Equation (eqn:dp) solves the Matrix-Free Dense Jacobian Chain Product Bracketing problem.

Figures (2)

  • Figure 1: Simple Example: Labeled DAG (a); primal (b); scalar tangent (c); scalar adjoint (d)
  • Figure 2: Tunnel flow simulation with OpenFOAM: Evolution of the (vertex) cut size of the DAG over potential split positions determined by the structure of target code.

Theorems & Definitions (2)

  • Theorem 3.1
  • Theorem 4.1