The Complex Gradient Operator and the CR-Calculus
Ken Kreutz-Delgado
TL;DR
This paper develops a comprehensive CR-calculus (Wirtinger) framework to optimize real-valued functions of complex vectors, addressing the limitations of classic complex derivatives for nonanalytic costs. It introduces real-derivatives with respect to z and ar{z}, and multi-representation formalisms (z, r, c-real, c-complex) to derive consistent first- and second-order expansions, gradients, and Hessians. The work presents multivariate extensions, transformation laws, and the gradient operator ∇_z, alongside Newton, Gauss-Newton, and quasi-Newton strategies tailored to complex optimization, with careful attention to admissibility and numerical stability. Applications include nonlinear least-squares and the complex LMS algorithm, illustrating exact solvability and robust gradient-based learning in complex domains. Overall, the CR-calculus provides a rigorous, transferable toolkit for efficient and stable optimization of real-valued objectives over complex parameters, with direct relevance to signal processing and communications.
Abstract
A thorough discussion and development of the calculus of real-valued functions of complex-valued vectors is given using the framework of the Wirtinger Calculus. The presented material is suitable for exposition in an introductory Electrical Engineering graduate level course on the use of complex gradients and complex Hessian matrices, and has been successfully used in teaching at UC San Diego. Going beyond the commonly encountered treatments of the first-order complex vector calculus, second-order considerations are examined in some detail filling a gap in the pedagogic literature.