Towards Intuitive Drone Operation Using a Handheld Motion Controller

TL;DR

This work addresses the need for intuitive, cross‑platform drone control by introducing a handheld motion controller that maps hand movements to the four primary UAV inputs via a simple yet robust interface built on the ESP32 and ExpressLRS. The system supports real‑world piloting with FPV drones and simulated environments through Bluetooth, enabling mixed reality training scenarios and integration with various simulators and VR/MR setups. A preliminary user study with ten novices demonstrates strong pragmatic and hedonic usability (mean Pragmatic 2.2, Hedonic 2.3 on the UEQ‑S), suggesting the interface lowers the entry barrier for drone piloting in education and research. Overall, the open‑protocol, gesture‑based controller advances human‑drone interaction by offering a portable, versatile, and interoperable solution for training, racing, and experimentation.

Abstract

We present an intuitive human-drone interaction system that utilizes a gesture-based motion controller to enhance the drone operation experience in real and simulated environments. The handheld motion controller enables natural control of the drone through the movements of the operator's hand, thumb, and index finger: the trigger press manages the throttle, the tilt of the hand adjusts pitch and roll, and the thumbstick controls yaw rotation. Communication with drones is facilitated via the ExpressLRS radio protocol, ensuring robust connectivity across various frequencies. The user evaluation of the flight experience with the designed drone controller using the UEQ-S survey showed high scores for both Pragmatic (mean=2.2, SD = 0.8) and Hedonic (mean=2.3, SD = 0.9) Qualities. This versatile control interface supports applications such as research, drone racing, and training programs in real and simulated environments, thereby contributing to advances in the field of human-drone interaction.

Figures (6)

• Figure 1: An operator during the drone piloting using a developed motion controller.
• Figure 2: A prototype of the developed handheld drone controller.
• Figure 3: An illustration of the drone control principle.
• Figure 4: Drone control through MR application: operator's view from the HMD (left), operator wearing the HMD and with handheld controller (right).
• Figure 5: (a) Flight area with a playground for drone control exercises for the experiment. (b) A participant during the drone piloting experiment.