PEERNet: An End-to-End Profiling Tool for Real-Time Networked Robotic Systems

TL;DR

PEERNet is introduced, an end-to-end and real-time profiling tool for cloud robotics that reveals non-intuitive behavior in robotic systems, such as asymmetric network transmission and bimodal language model output.

Abstract

Networked robotic systems balance compute, power, and latency constraints in applications such as self-driving vehicles, drone swarms, and teleoperated surgery. A core problem in this domain is deciding when to offload a computationally expensive task to the cloud, a remote server, at the cost of communication latency. Task offloading algorithms often rely on precise knowledge of system-specific performance metrics, such as sensor data rates, network bandwidth, and machine learning model latency. While these metrics can be modeled during system design, uncertainties in connection quality, server load, and hardware conditions introduce real-time performance variations, hindering overall performance. We introduce PEERNet, an end-to-end and real-time profiling tool for cloud robotics. PEERNet enables performance monitoring on heterogeneous hardware through targeted yet adaptive profiling of system components such as sensors, networks, deep-learning pipelines, and devices. We showcase PEERNet's capabilities through networked robotics tasks, such as image-based teleoperation of a Franka Emika Panda arm and querying vision language models using an Nvidia Jetson Orin. PEERNet reveals non-intuitive behavior in robotic systems, such as asymmetric network transmission and bimodal language model output. Our evaluation underscores the effectiveness and importance of benchmarking in networked robotics, demonstrating PEERNet's adaptability. Our code is open-source and available at github.com/UTAustin-SwarmLab/PEERNet.

Figures (6)

• Figure 2: PEERNet is an end-to-end profiling tool for networked robotics. Our modular tool allows users to inject custom external modules and specify custom configurations of sensors, networks, deep-learning pipelines and devices. Detailed profiles of latency, complete with asymmetric network timing, are generated.
• Figure 3: PEERNet's one-way delay measurements incur minimal error, of $2.01$ms. Latency estimates exhibit a constant offset error, measured here to be $-2.01$ms, and throughput estimates show decreasing variance for higher payload sizes, in the scale of images.
• Figure 4: PEERNet precisely quantifies inference costs at the edge and in the cloud. For the EfficientNetV2 family of models, local computation on an edge device is roughly 2.5 times as slow as offloaded computation to a cloud server, but cloud servers display a high variance in inference latency.
• Figure 5: Profiling with PEERNet reveals non-intuitive behavior of a vision language model. Responses to a single prompt vary in length, with two centers in the distribution of output length. Consequently, the inference latency is bimodal.
• Figure 6: Physical setup for Teleoperation. The objective of teleoperation is for the robot arm to manipulate the pen into the wooden cup, using visual information from two cameras, one on the arm, and another on a tripod.