Word2Wave: Language Driven Mission Programming for Efficient Subsea Deployments of Marine Robots

TL;DR

Word2Wave presents a language-driven framework for real-time AUV mission programming using seven atomic commands and a T5-Small Seq2Seq model. A GPT-based prompt-engineered data generator and a 2D Leaflet-based mission visualizer enable natural language to waypoint generation with real-time validation. Quantitative and user studies show that W2W delivers comparable accuracy with significantly faster inference and a 10% programming time compared with traditional interfaces, with SUS 76.25. This work demonstrates the practicality of language-based HMIs for subsea deployments and outlines future directions toward hands-free, dialogue-enabled embodied mission reasoning and telerobotics.

Abstract

This paper explores the design and development of a language-based interface for dynamic mission programming of autonomous underwater vehicles (AUVs). The proposed `Word2Wave' (W2W) framework enables interactive programming and parameter configuration of AUVs for remote subsea missions. The W2W framework includes: (i) a set of novel language rules and command structures for efficient language-to-mission mapping; (ii) a GPT-based prompt engineering module for training data generation; (iii) a small language model (SLM)-based sequence-to-sequence learning pipeline for mission command generation from human speech or text; and (iv) a novel user interface for 2D mission map visualization and human-machine interfacing. The proposed learning pipeline adapts an SLM named T5-Small that can learn language-to-mission mapping from processed language data effectively, providing robust and efficient performance. In addition to a benchmark evaluation with state-of-the-art, we conduct a user interaction study to demonstrate the effectiveness of W2W over commercial AUV programming interfaces. Across participants, W2W-based programming required less than 10\% time for mission programming compared to traditional interfaces; it is deemed to be a simpler and more natural paradigm for subsea mission programming with a usability score of 76.25. W2W opens up promising future research opportunities on hands-free AUV mission programming for efficient subsea deployments.

Figures (6)

• Figure 1: The proposed Word2Wave (W2W) framework offers a novel user interface for real-time AUV mission programming with natural human language. We demonstrate the effectiveness of W2W over traditional interfaces by subsea mission deployments on a NemoSens AUV platform.
• Figure 2: Demonstration of a sample mission programmed using the proposed Word2Wave (W2W) interface.
• Figure 3: The W2W-generated mission map (top) and the corresponding map loaded on the AUV mission interface (bottom) are visualized; The W2W mission text used to generate this is: "Start at $38.7969\degree$ N, $75.1538\degree$ W, Circle for a turn at a radius of $10$ m in a clockwise direction at an altitude of $1$ m. Move south $30$ m and then Move south $10$ m. Move south for $100$ m and then Track left for $100$ m at a spacing of $14$ m. End at $38.7968\degree$ N, $75.1535\degree$ W".
• Figure 4: Outlines for dataset preparation, model training, and deployment processes of the W2W framework are shown.
• Figure 5: Qualitative performance of our T5-Small mission generation engine for the $7$ language commands in W2W.