DiffTune: Auto-Tuning through Auto-Differentiation

TL;DR

DiffTune addresses auto-tuning for nonlinear robotic controllers by formulating tuning as a differentiable parameter optimization problem and using end-to-end gradient information. When data comes from a physical system, it employs sensitivity propagation to compute gradients despite a broken computation graph, and augments the controller with \,mathcal{L}_1 adaptive control to mitigate model uncertainties. The method is implemented in an open-source toolset and validated in simulation (Dubin's car and quadrotor) and real- world quadrotor experiments, showing faster convergence and up to a 3.5x reduction in tracking error with limited trials. This approach enables stable, efficient, and data-compatible online tuning that reduces sim-to-real gaps and improves practical robotic performance, especially in high-dimensional controller parameter spaces.

Abstract

The performance of robots in high-level tasks depends on the quality of their lower-level controller, which requires fine-tuning. However, the intrinsically nonlinear dynamics and controllers make tuning a challenging task when it is done by hand. In this paper, we present DiffTune, a novel, gradient-based automatic tuning framework. We formulate the controller tuning as a parameter optimization problem. Our method unrolls the dynamical system and controller as a computational graph and updates the controller parameters through gradient-based optimization. The gradient is obtained using sensitivity propagation, which is the only method for gradient computation when tuning for a physical system instead of its simulated counterpart. Furthermore, we use $\mathcal{L}_1$ adaptive control to compensate for the uncertainties (that unavoidably exist in a physical system) such that the gradient is not biased by the unmodelled uncertainties. We validate the DiffTune on a Dubin's car and a quadrotor in challenging simulation environments. In comparison with state-of-the-art auto-tuning methods, DiffTune achieves the best performance in a more efficient manner owing to its effective usage of the first-order information of the system. Experiments on tuning a nonlinear controller for quadrotor show promising results, where DiffTune achieves 3.5x tracking error reduction on an aggressive trajectory in only 10 trials over a 12-dimensional controller parameter space.

Figures (14)

• Figure 1: Illustration of an unrolled dynamical system as a computational graph.
• Figure 2: Difference between a conceptual dynamical system (a) and a physical system (b). Since a physical system in (b) does not have a complete computational graph for autodifferentiation, sensitivity propagation is used for gradient computation and is illustrated in (c).
• Figure 3: Comparison of auto-tuning of Dubin's car using DiffTune, AutoTune loquercio2022autotune, SafeOpt berkenkamp2016safe, and GIBO GIBO. The shaded areas show the range of RMSEs (min to max) achieved in a total of 10 runs of algorithms that involve stochasticity (AutoTune and GIBO).
• Figure 4: Comparison of performance between tuned and untuned controller parameters on testing trajectories for generalization.
• Figure 5: Loss $L$ subject to uncertainties (additive force $0.1 a_1 \sin(t)$ and moment $0.1 a_2 \cos(t)$ with $a_1$ and $a_2$ taking values from 1 to 10) in the ablation study of DiffTune and $\mathcal{L}_1$AC.

Theorems & Definitions (4)

• Remark 1
• Remark 2
• Remark 3
• Remark 4