Seasons's Greetings by AD

TL;DR

Problem addressed: show how type-generic AD can implement tangent, adjoint, and second-order analyses for simple scalar functions in C++. The paper presents two exemplar functions: $f({\bf x})=\sum x_{2i} x_{2i+1}$ and $f({\bf x})=\tfrac{1}{6}\sum x_i^3$, together with their first- and second-order derivatives. The key contributions are explicit templates for $f$, $f^{(1)}$, $f_{(1)}$, $f^{(1,2)}$, and $f_{(1)}^{(2)}$, plus sparsity-aware and direct-compression techniques to extract Hessian information efficiently. The results show that adjoint and compressed second-order approaches deliver substantial speedups on large $N$, while tangent modes incur larger costs, illustrating practical trade-offs. The work provides portable, technique-focused patterns for verifying and benchmarking AD in C++ with char arithmetic, and its code is available on GitHub.

Abstract

We use Algorithmic Differentiation (AD) to implement type-generic tangent and adjoint versions of $$ y=\sum_{i=0}^{n-1} x_{2 i} \cdot x_{2 i+1} $$ in C++. We run an instantiation for char-arithmetic and we print the gradient at $(101~77~114~114~32~121~109~88~115~97)^T$ to std::cout, yielding the output ``Merry Xmas''. Similar instantiations of type-generic second-order tangent and second-order adjoint versions of $$ y=\frac{1}{6} \cdot \sum_{i=0}^{n-1} x^3_{i} $$ yield ``Happy 2026'' at $(72~97~112~112~121~32~50~48~50~54)^T.$ Prepend a sufficiently large number of zeros to the input vector to explore the varying run times of the different derivative codes. The entire source code can be found on https://github.com/un110076/SeasonsGreetings.