A Multi-Robot Platform for Robotic Triage Combining Onboard Sensing and Foundation Models

TL;DR

The work proposes a heterogeneous air-ground robotic system for remote primary triage in mass-casualty incidents, integrating UAV-based victim localization with UGV-based vitals and injury assessment using a mix of unimodal and foundation-model analytics. Key contributions include a modular hardware-software stack, HDR low-light perception, onboard vision-language and localization pipelines (LLaVA, NVILA-Lite-2B, Grounding DINO, DINOv3), and multi-modal vitals estimation (rPPG, MTTS-CAN, mmWave, LWIR, PCR). The system demonstrates end-to-end triage capabilities within the DARPA Triage Challenge, highlighting onboard inference improvements, data sharing via MOCHA, and operator interfaces (ATAK, browser-based UGV oversight). The results indicate significant potential to augment first responders by providing timely casualty localization, vital signs, and injury assessments with reduced risk to human responders, while identifying routes for data-centric and autonomy-focused future work.

Abstract

This report presents a heterogeneous robotic system designed for remote primary triage in mass-casualty incidents (MCIs). The system employs a coordinated air-ground team of unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) to locate victims, assess their injuries, and prioritize medical assistance without risking the lives of first responders. The UAV identify and provide overhead views of casualties, while UGVs equipped with specialized sensors measure vital signs and detect and localize physical injuries. Unlike previous work that focused on exploration or limited medical evaluation, this system addresses the complete triage process: victim localization, vital sign measurement, injury severity classification, mental status assessment, and data consolidation for first responders. Developed as part of the DARPA Triage Challenge, this approach demonstrates how multi-robot systems can augment human capabilities in disaster response scenarios to maximize lives saved.

Figures (7)

• Figure 1: Our heterogeneous robot team for triage during a mass casualty simulation at the Guardian Centers, Perry, GA in March 2025.
• Figure 2: These diagrams outline the hardware setup for both the UAV (a) and UGV (b). The sensors (green) connect to the onboard computer via USB. Each robot is equipped with a Rajant radio connected via ethernet (blue arrow), and other communication hardware (yellow) is there for control and safety. Finally, power (red) is provided from a battery to the onboard computer via a power distribution board. The dotted outline on the UGV indicates modularity, everything within the box can be taken and placed on another robot and only one connection to that new robot is needed.
• Figure 3: This depicts how the software is laid out on each platform. The hardware level consists of the sensors, the onboard computer and the robot itself. All of our software runs on the host OS (yellow) of the onboard computer. All software is modularized into docker images (blue), which we depict in two levels on the UAV and UGV. The system base level takes care of hardware interfaces and data management. The top layer is for victim assessment. The arrows depict the flow of data starting at the sensor hardware.
• Figure 4: System Monitoring Interface from the DARPA Triage Challenge, held at Guardian Centers, Perry, GA in October 2025.
• Figure 5: Preprocessing image using YOLOv8, HSV Skin Extraction Results and RGB Skin Extraction. (\ref{['fig:original']}) Original image of a person sitting on the grass. (\ref{['fig:cropped']}) Cropped image using YOLOv8. (\ref{['fig:skin']}) Extracted skin image in HSV color space. (\ref{['fig:rgbskin']}) Extracted skin image in RGB color space, illustrating the difficulty of skin isolation in RGB.