NonlinearSolve.jl: High-Performance and Robust Solvers for Systems of Nonlinear Equations in Julia

TL;DR

NonlinearSolve.jl addresses the challenge of solving nonlinear systems defined by $f(u, θ)=0$ across scientific domains. It introduces a Julia-native framework with a unified API, automatic solver selection, GPU-accelerated static kernels, and sparse automatic differentiation, enabling scalable performance for large problems. The paper demonstrates robustness and speed through benchmarks against PETSc SNES, Sundials KINSOL, MINPACK, and NLsolve, and showcases applicability to DFN battery initialization and large ill-conditioned Brusselator systems. This work enables scalable, flexible nonlinear solving within the SciML ecosystem, unlocking new modeling and simulation capabilities across disciplines.

Abstract

Efficiently solving nonlinear equations underpins numerous scientific and engineering disciplines, yet scaling these solutions for challenging system models remains a challenge. This paper presents NonlinearSolve.jl -- a suite of high-performance open-source nonlinear equation solvers implemented natively in the Julia programming language. NonlinearSolve.jl distinguishes itself by offering a unified API that accommodates a diverse range of solver specifications alongside features such as automatic algorithm selection based on runtime analysis, support for GPU-accelerated computation through static array kernels, and the utilization of sparse automatic differentiation and Jacobian-free Krylov methods for large-scale problem-solving. Through rigorous comparison with established tools such as PETSc SNES, Sundials KINSOL, and MINPACK, NonlinearSolve.jl demonstrates robustness and efficiency, achieving significant advancements in solving nonlinear equations while being implemented in a high-level programming language. The capabilities of NonlinearSolve.jl unlock new potentials in modeling and simulation across various domains, making it a valuable addition to the computational toolkit of researchers and practitioners alike.

Figures (12)

• Figure 1: NonlinearSolve.jl allows seamless switching between different line search routines. We can specify HagerZhang(), BackTracking(), MoreTheunte(), StrongWolfe(), or LiFukushimaLineSearch() to opt-in to using line search. Line search enables NewtonRaphson to converge faster on the generalized Rosenbrock problem [\ref{['eq:gen_rosenbrock_function']}], while it fails to converge without a line search.
• Figure 2: Work-Precision Diagrams for Two Instances of the 23 Test Problems: Trust-region radius update schemes directly affect the convergence of the algorithm, NonlinearSolve.jl implements various update schemes and allows switching them with a unified API.
• Figure 3: NonlinearSolve.jl has a modular architecture with the core building blocks: Jacobian Computation Algorithm, Globalization Strategy, and Descent Direction. By combining these fundamental components, NonlinearSolve.jl facilitates creating diverse and powerful solver algorithms tailored to specific problem characteristics. This composability feature underscores the framework's versatility in tackling a wide range of nonlinear equations, showcasing the potential for innovative algorithm development through the flexible integration of different strategies and techniques.
• Figure 4: Default Nonlinear Solve Poly-Algorithm focused on balancing speed and robustness. NonlinearSolve.jl first tries less robust Quasi-Newton methods for better performance and then tries more robust techniques if the faster ones fail. Based on the choice of linear solver, these methods automatically switch to a Jacobian-free variant.
• Figure 5: Decision Flowchart for Default Linear Solver Selection: Starting with the determination of the Jacobian type (e.g., square, rectangular, symmetric, structured), the flowchart guides through various conditions such as sparsity and conditioning to select the most appropriate solver. Options range from specialized solvers like KLU for sparse matrices to robust methods like SVD for extremely ill-conditioned cases. This adaptive approach ensures that NonlinearSolve.jl utilizes the most efficient linear solver, enhancing performance and accuracy for a wide array of nonlinear problems.