Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Derivative-free discrete gradient methods

Håkon Noren Myhr, Sølve Eidnes

TL;DR

This work tackles preserving first integrals in ODEs without derivatives by introducing a derivative-free, fourth-order discrete gradient method (DGM) built from a symmetrized Itoh–Abe DG and finite-difference approximations. By aligning order-theory with carefully constructed $S_4(x,\hat{x},h)$ and derivative-free approximations $D_2^{\tau}\overline{\nabla}_{\text{SIA}} H$ and $\nabla^2_{\tau} H$, the authors derive an implicit scheme solved via inexact Newton iterations and provide convergence bounds that balance finite-difference and machine-precision errors. They verify the theory with numerical experiments on a double pendulum, Lennard–Jones oscillator, and a topographic-Hamiltonian, showing that derivative-free variants achieve comparable accuracy to AD-based methods but can be roughly 50 times faster, highlighting the method’s practicality for expensive or inaccessible gradients. The results suggest useful applications in training Hamiltonian neural networks and in gradient-free sampling such as Conservative Hamiltonian Monte Carlo, where gradient computations can be a bottleneck.

Abstract

Discrete gradient methods are a class of numerical integrators producing solutions with exact preservation of first integrals of ordinary differential equations. In this paper, we apply order theory combined with the symmetrized Itoh--Abe discrete gradient and finite differences to construct an integral-preserving fourth-order method that is derivative-free. The numerical scheme is implicit and a convergence result for Newton's iterations is provided, taking into account how the error due to the finite difference approximations affects the convergence rate. Numerical experiments verify the order and show that the derivative-free method is significantly faster than obtaining derivatives by automatic differentiation. Finally, an experiment using topographic data as the potential function of a Hamiltonian oscillator demonstrates how this method allows the simulation of discrete-time dynamics from a Hamiltonian that is a combination of data and analytical expressions.

Derivative-free discrete gradient methods

TL;DR

This work tackles preserving first integrals in ODEs without derivatives by introducing a derivative-free, fourth-order discrete gradient method (DGM) built from a symmetrized Itoh–Abe DG and finite-difference approximations. By aligning order-theory with carefully constructed and derivative-free approximations and , the authors derive an implicit scheme solved via inexact Newton iterations and provide convergence bounds that balance finite-difference and machine-precision errors. They verify the theory with numerical experiments on a double pendulum, Lennard–Jones oscillator, and a topographic-Hamiltonian, showing that derivative-free variants achieve comparable accuracy to AD-based methods but can be roughly 50 times faster, highlighting the method’s practicality for expensive or inaccessible gradients. The results suggest useful applications in training Hamiltonian neural networks and in gradient-free sampling such as Conservative Hamiltonian Monte Carlo, where gradient computations can be a bottleneck.

Abstract

Discrete gradient methods are a class of numerical integrators producing solutions with exact preservation of first integrals of ordinary differential equations. In this paper, we apply order theory combined with the symmetrized Itoh--Abe discrete gradient and finite differences to construct an integral-preserving fourth-order method that is derivative-free. The numerical scheme is implicit and a convergence result for Newton's iterations is provided, taking into account how the error due to the finite difference approximations affects the convergence rate. Numerical experiments verify the order and show that the derivative-free method is significantly faster than obtaining derivatives by automatic differentiation. Finally, an experiment using topographic data as the potential function of a Hamiltonian oscillator demonstrates how this method allows the simulation of discrete-time dynamics from a Hamiltonian that is a combination of data and analytical expressions.
Paper Structure (17 sections, 3 theorems, 56 equations, 7 figures, 1 table)

This paper contains 17 sections, 3 theorems, 56 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Discrete gradient methods
  3. Derivative-free numerical algorithms
  4. Derivative-free discrete gradient methods
  5. Finite difference approximations
  6. Hessian approximation
  7. Derivative of the SIA discrete gradient
  8. Finite precision in difference schemes
  9. Derivative-free fourth-order DGM
  10. Convergence of Newton's method
  11. Computational complexity
  12. Numerical experiments
  13. Convergence of $S_4^{\tau}$, $F_{\tau}$ and $F'_{\tau}$
  14. Convergence of numerical solution
  15. Implementation details
  16. ...and 2 more sections

Key Result

Theorem 2.1

Consider the Hamiltonian system eq:hamiltonian_ODE. Let $\overline \nabla H$ be a second-order symmetric discrete gradient of $H$, and $S_4(x,\hat{x},h)$ the skew-symmetric matrix with $\bar{x} := \frac{x+\hat{x}}{2}$ and where $D_2 \overline{\nabla}H(x,\hat{x})$ is the derivative of the discrete gradient with respect to the second argument. Then the discrete gradient method is of fourth order.

Figures (7)

  • Figure 1: Numerical and theoretical convergence of $S_{\tau}$, $F_{\tau}$ and $F'_{\tau}$ when the step size in time $h$ decreases. Dotted lines represent the theoretical convergence rates (Lemma \ref{['lem:err_in_S']} and Theorem \ref{['thm:df_dgm_4_newton']}) and the solid lines the numerical results.
  • Figure 2: Error results for the double pendulum (left column) and the Lennard--Jones oscillator (right column). The top row shows the global error at end time, $\|x_N - x(T)\|_2$, plotted against the step size $h$. The bottom row shows the $L_2$ error over time, $\|x_n - x(t_n)\|_2$, where the smallest step size was used. Dashed lines display the derivative-free (DF) methods. The gray dotted lines represent order $h^1$, $h^2$ and $h^4$ respectively.
  • Figure 3: Energy error for the double pendulum (left) and the Lennard--Jones oscillator (right) plotted over time where the smallest step size was used. Dashed lines represent the derivative-free (DF) methods.
  • Figure 4: Numbers of evaluations of $H(x)$ for the double pendulum (left) and the Lennard--Jones oscillator (right) over multiple experiments with different $h$.
  • Figure 5: Work-precision diagrams where the $L_2$ (top row) and energy error (bottom row) is plotted against computational time for the double pendulum (left column) and the Lennard--Jones oscillator (right column) over multiple experiments with different step size in time $h$. Dashed lines display the derivative-free (DF) methods.
  • ...and 2 more figures

Theorems & Definitions (8)

  • Theorem 2.1: Fourth-order DGM
  • proof
  • Remark 2.2
  • Lemma 3.1
  • proof
  • Remark 3.2
  • Theorem 4.1
  • proof