Neural-Fly Enables Rapid Learning for Agile Flight in Strong Winds

TL;DR

Neural-Fly tackles safe, agile UAV flight in dynamic winds by learning a wind-invariant representation of aerodynamics and coupling it with online composite adaptation to update wind-specific effects. The offline phase uses DAIML to train a wind-agnostic basis $\phi$ and wind-dependent coefficients $a(w)$ from only 12 minutes of flight data, with spectral normalization and a domain discriminator to guarantee invariance. The online phase employs a regularized composite adaptation to achieve rapid, stable tracking under unseen wind conditions, with formal exponential stability guarantees and validation in Caltech’s wind tunnel up to $12.1\,\text{m/s}$ and outdoor tests. Compared to state-of-the-art nonlinear and adaptive controllers, Neural-Fly delivers substantial tracking accuracy gains, transfers across drones, and maintains robustness with modest onboard computation, enabling practical deployment on standard UAV hardware.

Abstract

Executing safe and precise flight maneuvers in dynamic high-speed winds is important for the ongoing commoditization of uninhabited aerial vehicles (UAVs). However, because the relationship between various wind conditions and its effect on aircraft maneuverability is not well understood, it is challenging to design effective robot controllers using traditional control design methods. We present Neural-Fly, a learning-based approach that allows rapid online adaptation by incorporating pretrained representations through deep learning. Neural-Fly builds on two key observations that aerodynamics in different wind conditions share a common representation and that the wind-specific part lies in a low-dimensional space. To that end, Neural-Fly uses a proposed learning algorithm, domain adversarially invariant meta-learning (DAIML), to learn the shared representation, only using 12 minutes of flight data. With the learned representation as a basis, Neural-Fly then uses a composite adaptation law to update a set of linear coefficients for mixing the basis elements. When evaluated under challenging wind conditions generated with the Caltech Real Weather Wind Tunnel, with wind speeds up to 43.6 kilometers/hour (12.1 meters/second), Neural-Fly achieves precise flight control with substantially smaller tracking error than state-of-the-art nonlinear and adaptive controllers. In addition to strong empirical performance, the exponential stability of Neural-Fly results in robustness guarantees. Last, our control design extrapolates to unseen wind conditions, is shown to be effective for outdoor flights with only onboard sensors, and can transfer across drones with minimal performance degradation.

Figures (11)

• Figure 1: Agile flight through narrow gates. (A) Caltech Real Weather Wind Tunnel system, the quadrotor UAV, and the gate. In our flight tests, the UAV follows an agile trajectory through narrow gates, which are slightly wider than the UAV itself, under challenging wind conditions. (B-C) Trajectories used for the gate tests. In (B), the UAV follows a figure-8 through one gate, with wind speed 3.1m/s or time-varying wind condition. In (C), the UAV follows an ellipse in the horizontal plane through two gates, with wind speed 3.1m/s. (D-E) Long-exposure photos (with an exposure time of 5s) showing one lap in two tasks. (F-I) High-speed photos (with a shutter speed of 1/200s) showing the moment the UAV passed through the gate and the interaction between the UAV and the wind.
• Figure 2: Offline meta-learning and online adaptive control design. (A) The online adaptation block in our adaptive controller. Our controller leverages the meta-trained basis function $\phi$, which is a wind-invariant representation of the aerodynamic effects, and uses composite adaptation (that is, including tracking-error-based and prediction-error-based adaptation) to update wind-specific linear weights $\hat{a}$. The output of this block is the wind-effect force estimate, $\hat{f}=\phi\hat{a}$. (B) The illustration of our meta-learning algorithm DAIML. We collected data from wind conditions $\{w_1,\cdots,w_K\}$ and applied Algorithm \ref{['alg:DIML']} to train the $\phi$ net. (C) The diagram of our control method, where the grey part corresponds to (A). Interpreting the learned block as an aerodynamic force allows it to be incorporated into the feedback control easily.
• Figure 3: Training data collection. (A) The xyz position along a two-minute randomized trajectory for data collection with wind speed 8.3km/h (3.7m/s), in the Caltech Real Weather Wind Tunnel. (B) A typical 10-second trajectory of the inputs (velocity, attitude quaternion, and motor speed PWM command) and label (offline calculation of aerodynamic residual force) for our learning model, corresponding to the highlighted part in (A). (C) Histograms showing data distributions in different wind conditions. (C) Left: distributions of the $x$-component of the wind-effect force, $f_x$. This shows that the aerodynamic effect changes as the wind varies. (C) Right: distributions of the pitch, a component of the state used as an input to the learning model. This shows that the shift in wind conditions causes a distribution shift in the input.
• Figure 4: t-SNE plots showing the evolution of the linear weights ($a^*$) during the training process. As the number of training epochs increases, the distribution of $a^*$ becomes more clustered with similar wind speed clusters near each other. The clustering also has a physical meaning: after training convergence, the right top part corresponds to a higher wind speed. This suggests that DAIML successfully learned a basis function $\phi$ shared by all wind conditions, and the wind-dependent information is contained in the linear weights. Compared to the case without the adversarial regularization term (using $\alpha=0$ in Algorithm \ref{['alg:DIML']}), the learned result using our algorithm is also more explainable, in the sense that the linear coefficients in different conditions are more disentangled.
• Figure 5: Depiction of the trajectory tracking performance of each controller in several wind conditions. The baseline nonlinear controller can track the trajectory well, however, the performance substantially degrades at higher wind speeds. INDI, ${\mathcal{L}_1}$, and Neural-Fly-Constant have similar performance and improve over the nonlinear baseline by estimating the aerodynamic disturbance force quickly. Neural-Fly and Neural-Fly-Transfer use a learned model of the aerodynamic effects and adapt the model in real time to achieve lower tracking error than the other methods.

Theorems & Definitions (12)

• Theorem 1
• Lemma 2
• Theorem 3
• proof
• Theorem 4
• proof
• Corollary 5
• proof
• Corollary 6
• proof